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1 Introduction 


Under  the  auspices  of  the  Department  of  Transportation  Intelligent  Vehicle  Initiative,  the 
National  Highway  Traffic  Safety  Administration  (NHTSA)  is  evaluating  the  safety 
benefits  of  Road  Departure  Crash  Warning  Systems  (RDCWS).  These  systems  warn 
inattentive  drivers  when  their  vehicle  is  about  to  depart  the  road  or  collide  with  a roadside 
obstacle.  To  further  understand  the  capabilities  and  benefits  of  these  warning  systems, 
the  NHTSA,  with  the  assistance  of  the  National  Institute  of  Standards  and  Technology 
(NIST),  is  devising  test  procedures  that  provide  performance  data  in  an  objective  and 
quantitative  manner.  These  tests  will  evolve  to  reflect  new  insight  into  performance 
measurement  and  to  maintain  pace  with  the  capabilities  of  the  warning  systems.  In  the 
long  term,  these  tests  may  one-day  become  part  of  a standard  procedure  developers 
follow  to  achieve  a safety  rating  similar  to  the  “5-star”  ratings  used  in  the  automotive 
industry  today. 

The  RDCWS  Field  Operation  Test  (FOT)  program  [1]  validated  many  of  the  tests 
included  in  this  report.  The  developers  of  the  FOT  RDCWS  used  an  early  version  of 
these  procedures  to  validate  system  performance.  The  developers  adapted  the  procedures 
to  meet  the  time,  funding  and  available  track  geometry  of  the  FOT.  The  RDCWS  FOT 
contractor  and  NIST  conducted  the  tests  (called  validation  tests)  at  the  Transportation 
Research  Center  (Maryville  Ohio)  from  October  1,  2003  through  April  28,  2004.  The 
successful  performance  of  the  system  during  the  tests  allowed  the  FOT  to  proceed  to 
Phase  2. 

These  tests  utilize  considerable  research  from  the  international  community,  NHTSA,  the 
Volpe  National  Transportation  Systems  Center,  and  NIST. 

1.  An  outline  of  the  NIST  RDCWS  research  results  appears  in  a final  report  titled 
“Run-Off-Road  Collision  Avoidance  Using  IVHS  Countermeasures”  [2]  (referred  to 
as  the  ROR  report  in  this  document) 

2.  The  Road  Departure  Crash  Warning  System  FOT  Request  for  Applicants  issued  by 
the  NHTSA  [1]  contains  the  performance  capabilities  expected  of  a warning  system 

3.  A NHTSA  and  Volpe  National  Transportation  Systems  Center  report  on  an  analysis 
of  crash  data  identifies  high-risk  crash  scenarios  [3]. 

4.  Reference  [4]  is  the  International  Standards  Organization  test  procedure  for  lane 
departure  warning  systems. 

A RDCWS  consists  of  the  sensor,  the  alert  logic  (correctness  and  timing)  and  the  driver 
interface.  The  tests  in  this  document  address  sensor  and  alert  logic  performance  only.  A 
complete  evaluation  of  the  driver  interface  would  consider  how  a driver  responds  to  the 
warning.  These  tests  only  evaluate  the  driver  interface  from  the  standpoint  of  whether 
the  interface  presented  the  warning  or  not. 
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The  methods  for  evaluation  of  the  warning  system  focus  on  two  areas:  correctness  and 
timing.  The  correctness  of  each  warning  issued  by  the  system  is  either  true  positive,  false 
positive,  false  negative  or  true  negative  (Table  1).  Statistical  evaluations,  such  as  efficacy 
rate  (the  percentage  of  warnings  that  produce  the  desired  effects)  and  false  alarm  rate, 
summarize  the  results  of  the  ratings. 


System  Response 

Situations  Requiring  a 
Warning 

Situations  in  Which  a 
Warning  is  not  Required 

Warning 

True  Positive  (TP) 
Correctly  warns  of  road 
departure 

False  Positive  (FP) 

False  alarm 

No  Warning 

False  Negative  (FN) 
Misses  road  departure 

True  Negative  (TN) 
Correctly  does  not  warn  of 
road  departure. 

Table  1 Rating  of  warning  system  signals. 


The  second  area  of  evaluation  examines  the  timing.  A warning  is  of  little  benefit  if  it 
does  not  provide  the  driver  with  sufficient  time  to  avoid  the  departure.  Equations  of 
motion  are  the  basis  of  the  performance  metrics  and  determine  the  time  afforded  to  a 
driver.  Time  limits  determine  whether  a warning  is  useful. 

The  methods  quantify  performance  but  do  not  specify  the  passing  grade  (e.g.,  the  system 
must  pass  90%  of  the  tests).  The  organization  interested  in  the  test  results  should  select 
the  pass/fail  criteria  (some  examples  are  provided  in  section  5). 

Evaluation  of  the  metrics  requires  real-time  measurements,  preferably  from  an 
independent  measurement  system  (IMS).  Using  the  warning  system  sensors  and 
processing  algorithms  may  bias  the  system  evaluation.  For  illustrative  purposes,  the  test 
procedures  include  instructions  for  taking  measurements  using  an  IMS  developed  by 
NIST. 

The  ability  to  detect  the  road  edge  is  critical  to  the  success  of  a warning  system.  For 
basic  warning  systems,  the  road  edges  are  the  lane  markers  in  the  vehicle’s  current  travel 
lane  and  the  system  issues  a warning  before  the  vehicle  enters  the  adjacent  lane. 
However,  driver  behavior  can  produce  slight  lane  excursions  and  warnings  for  these 
instances  might  become  a nuisance.  The  intent  of  a road  departure  crash  warning  system 
is  to  detect  true  crash  conditions,  such  as  driving  off-road  or  drifting  into  obstacles 
outside  the  current  lane.  The  objective  to  limit  nuisance  alarms  and  to  focus  on  real 
crashes  led  to  the  concept  of  an  available  maneuver  room  (AMR),  which  is  the  region 
outside  the  lane  marker  that  is  drivable  (e.g.,  paved  shoulder)  and  clear  of  obstacles.  The 
AMR  added  to  the  outer  lane  boundary  defines  the  road  boundary. 

Tests  evaluate  warning  system  performance  for  different  types  of  road-edges.  The  first 
set  of  tests  examines  the  system’s  ability  to  detect  a road  edge  defined  by  the  outer  lane 
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marker  under  the  assumption  that  there  is  a minimal  amount  of  AMR.  These  tests  are 
appropriate  for  a basic  lane  departure  system. 

The  next  test  set  examines  the  system's  ability  to  detect  a road  edge  defmed  by  the 
transition  from  pavement  to  off-road.  The  size  of  the  shoulder  may  vary  considerably, 
which  would  lead  to  a wide  range  of  warning  locations.  Previous  work  [2]  suggests  a 
need  for  consistent  warning  location.  Therefore,  the  AMR  has  a default  maximum  value 
regardless  of  the  shoulder  width. 


Another  test  set  examines  the  system’s  ability  to  detect  a road  edge  defined  by  an 
obstacle  on  the  shoulder.  In  one  case,  the  vehicle  drifts  into  an  obstacle  adjacent  to  the 
vehicle,  such  as  a Jersey  barrier  or  guardrail.  In  the  other  case,  the  vehicle  drifts  into  an 
obstacle  up  ahead,  such  as  a car  parked  on  the  shoulder.  In  either  case,  the  road  edge  is 
an  imaginary  line  at  the  base  of  the  obstacle  and  parallel  to  the  lane  marker. 

The  tests  above  address  lateral  drift  scenarios.  The  final  set  of  tests  evaluates  system 
performance  when  the  vehicle  is  traveling  too  fast  for  an  approaching  curve,  called  the 
curve-speed  scenario. 

This  document  has  nine  sections.  The  first  section  is  this  introduction.  Section  2 contains 
definitions  of  the  terminology  used  throughout  the  document.  Section  3 contains  an 
overview  of  the  general  test  structure  common  to  each  test.  Section  4 describes  a 
measurement  system  to  independently  collect  data  for  analyzing  warning  system 
performance.  Section  5 describes  evaluation  procedures  that  are  commonly  used  in  lateral 
drift  and  curve  speed  tests.  Section  6 contains  tests  to  evaluate  departures  on  roads  with 
no  available  maneuver  room.  Section  7 contains  tests  to  evaluate  departures  on  roads  with 
available  maneuver  room.  Section  8 contains  tests  to  evaluate  departures  towards  barriers 
and  obstacles.  Section  9 contains  tests  to  evaluate  on  curves  due  to  excessive  speed. 
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2 Definitions 

This  section  contains  definitions  of  terms  used  in  the  tests.  The  definitions  fall  into  one 
of  the  following  categories:  road,  vehicle,  crash  and  environmental. 

2.1  Road  Definitions 

2.1.1  Road 

The  road  consists  of  all  travel  lanes  in  the  same  direction  of  travel.  This  includes  paved 
shoulders  clear  of  obstacles.  Roads  may  be  divided  or  undivided. 

2.1.2  Undivided  Road 

An  undivided  road  is  a road  with  two-way  traffic  (see  Figure  1).  Either  double  lane 
markers  or  a single  dashed  lane  marker  separates  the  directions  of  travel.  A single  dashed 
lane  marker  indicates  passing  in  either  direction.  Double  lane  markers  consist  of 
combinations  of  solid  and  dashed  stripes.  Dashed  indicates  passing  is  permitted  and  solid 
prohibits  passing. 


road 


off 

road 


Obstacle 


Figure  1.  Undivided  road  includes  all  travel  lanes  and  the  shoulder  (s) 


2.1.3  Divided  Road 

A divided  road  is  a road  with  one-way  traffic  (see  Figure  2).  Dashed  white  lane  markers 
separate  lanes  of  traffic. 
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Figure  2.  Divided  road  includes  all  travel  lanes  and  the  shoulder  (s) 


2.1.4  Lane 

The  lane  is  the  area  of  paved  surface  between  two  parallel  lane  markings.  The  distance 
between  the  inside  edges  of  the  lane  markings  should  be  3.7  m (12  ft).  The  lane  must 
have  markings  on  the  left  side.  If  the  lane  has  no  markings  on  the  right  side  (worn  or 
missing),  then  the  lane  is  3.7  m measured  from  the  inside  edge  of  the  left  side  lane 
marker. 

2.1.5  Shoulder 

The  shoulder  is  the  portion  of  drivable  surface  to  the  right  of  an  undivided  road  and  on 
either  side  of  a divided  road.  Refer  to  s in  Figure  1 and  Figure  2. 

2.1.6  Available  Maneuver  Room 

The  available  maneuver  room  is  the  region  of  the  road  outside  the  outer  lane  marker  that 
is  safe  to  drive  (see  AMR  in  Figure  1 and  Figure  2).  Obstacles  on  the  shoulder  or 
absences  of  shoulders  limit  the  AMR.  By  design,  the  AMR  does  not  exceed  a default 
maximum  value  ( 1 m),  regardless  of  the  size  of  the  shoulder.  The  default  minimum 
value  is  15  cm  (6  in). 

2.1.7  Road  Boundary 

The  road  boundary  (also  referred  to  as  road  edge)  is  the  real  or  imaginary  boundary  line 
on  the  pavement  and  defines  the  edge  of  the  road  (see  Figure  1 and  Figure  2).  The  road 
boundary  is  parallel  to  the  outer  lane  marker  and  extends  beyond  the  outer  lane  marker  by 
the  available  maneuver  room  distance.  A warning  system  senses  the  available  maneuver 
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room  (e.g.,  obstacles  on  the  shoulder)  and  adjusts  the  road  boundary  accordingly.  The 
warning  system  also  uses  the  distance  to  the  road  boundary  to  calculate  when  to  issue  a 
warning. 

2.1.8  Curved  Road 

A curved  road  is  a section  of  road  with  a radius  less  than  1 000  m. 

2.1.9  Straight  Road 

A straight  road  is  a section  of  road  with  a radius  greater  than  1 000  m. 

2.1.10  Lane  Markings 

The  definitions  for  lane  markings  come  from  part  three  of  the  Manual  on  Uniform  Traffic 
Control  Devices  (http://mutcd.fliwa.dot.gov/).  Lane  marker  dimensions  for  the  test  are: 


Lane  marker  value 

Dimensions 

Maker  width 

10  cm  to  15  cm  (4  in  to  6 in) 

Dashed  marker  length 

3 m ( 1 0 ft)  ± 5 % 

Dash  marker  gap  length  (distance  between  dashed 
markers) 

9 m (30  ft)  ± 5 % 

Appendix  A “Methods  for  Determining  Road  Dimensions  and  Geometry”  contains  the 
recommended  method  to  verify  the  lane  marker  dimensions. 

The  manual  states: 

Centerline  pavement  markings,  when  used,  shall  be  the  pavement  markings  used  to 
delineate  the  separation  of  traffic  lanes  that  have  opposite  directions  of  travel  on  a 
roadway  and  shall  be  yellow. 

The  centerline  markings  on  two-lane,  two-way  roadways  shall  be  one  of  the 
following  as  shown  in  Figure  3B-1: 

A.  Two-direction  passing  zone  markings  consisting  of  a normal  broken  yellow  line 
where  crossing  the  centerline  markings  for  passing  with  care  is  permitted  for  traffic 
traveling  in  either  direction; 

B.  One-direction  no-passing  zone  markings  consisting  of  a normal  broken  yellow 
line  and  a normal  solid  yellow  line  where  crossing  the  centerline  markings  for 
passing  with  care  is  permitted  for  the  traffic  traveling  adjacent  to  the  broken  line, 
but  is  prohibited  for  traffic  traveling  adjacent  to  the  solid  line;  and 

C.  Two-direction  no-passing  zone  markings  consisting  of  two  normal  solid  yellow 
lines  where  crossing  the  centerline  markings  for  passing  is  prohibited  for  traffic 
traveling  in  either  direction. 
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Figure  3B-1.  Examples  of  Two-Lane,  Two-Way  Marking  Applications 


a - Typical  two-lane,  two-way  marking 

with  passing  permitted  in  both  directions 


b -Typical  two-lane,  two-way  marking 
with  no  passing  zones 


-*  Direction  of  Isavei 


Note: 

See  Section  38.0? 
fcr  edge  line  Wa -rents 


When  used,  lane  line  pavement  markings  delineating  the  separation  of  traffic  lanes 
that  have  the  same  direction  of  travel  shall  be  white. 

Where  crossing  the  lane  line  markings  with  care  is  permitted,  the  lane  line  markings 
shall  consist  of  a normal  broken  white  line. 

Where  crossing  the  lane  line  markings  is  discouraged,  the  lane  line  markings  shall 
consist  of  a normal  solid  white  line. 

Where  crossing  the  lane  line  markings  is  prohibited,  the  lane  line  markings  shall 
consist  of  two  normal  solid  white  lines. 

If  used,  the  right  edge  line  pavement  markings  shall  consist  of  a normal  solid  white 
line  to  delineate  the  right  edge  of  the  roadway. 
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2.1.11  Lane  Marker  Reflectivity 

The  lane  marker  reflectivity  (or  brightness)  is  the  ratio  between  the  luminance  of  the 
marker  (light  coming  from  the  marker)  and  the  illuminance  on  the  marker  (the  light 
reaching  the  marker). 

In  daylight  conditions,  the  measure  of  reflectivity  is  the  coefficient  of  luminance  under 
diffuse  illumination  (Qd). 

In  nighttime  conditions,  the  brightness  is  a function  of  the  headlights  and  the 
retroreflectivity  of  the  lane  marker.  The  measure  of  retroreflectivity  is  the  coefficient  of 
retroreflected  luminance  (Rl). 

Both  of  these  coefficients  are  in  units  of  millicandelas  per  square  meter  per  lux  (mcd*nT 
Mx1).  The  geometry  regarding  the  orientation  of  the  light  source,  the  orientation  the 
viewer  and  the  distance  to  the  lane  marker  is  part  of  the  measurement  process  [9]. 


The  values  below  come  from  Table  1 in  reference  [14]. 


Lane  Marker 
Color 

Minimum  coefficient  under  diffuse 
illumination,  Qd  (mcd  m'2/lx) 

White 

<130  (for  Asphalt) 

< 1 60  (for  Concrete) 

Yellow 

< 100 

Appendix  B contains  the  recommended  method  to  verify  the  lane  marker  reflectivity. 

2.1.12  Lane  Marker  Color 

The  daytime  color  of  the  lane  marker  is  defined  by  its  chromaticity  and  luminance 
(x,y,Y).  The  CIE  (Commission  International  d'Eclairage,  or  International  Commission 
on  Illumination,  http://wwvvxie.co.at/cieA  chromaticity  chart  shows  the  coordinates  (x,y) 
of  visible  colors.  The  luminance  (Y)  defines  the  brightness  of  the  color. 

The  four  points  listed  in  the  table  below  bound  the  region  of  allowable  colors  for  both 
white  and  yellow  lane  markers.  Each  color  includes  the  luminance  factor  (Y).  The 
values  come  from  Table  1 and  Table  3 of  reference  [12]  . One  can  use  a commercial 
measurement  device  to  measure  color  properties.  


Lane  Marker 
Color 

Daytime  Chromaticity 
Coordinates  (x,y) 

Luminance  factor  Y (%) 

White 

0.355,0.355 

0.305,  0.305 

0.285,  0.325, 

0.335,  0.375 

Minimum  of  45 

Yellow 

0.560,  0.440 

0.490,  0.510 

0.420,  0.440, 

0.460,  0.400 

Minimum  of  30 
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Figure  3.  CIE  Chromaticity  Chart 


2.2  Vehicle  Definitions 

Several  definitions  follow  the  terminology  in  reference  [6]. 

2.2.1  Vehicle  Coordinate  Systems 

Figure  4 shows  the  vehicle  coordinate  system.  The  origin  of  this  system  lies  on  the  road 
directly  under  the  vehicle's  center  of  gravity.  The  z-axis  points  down  into  the  road. 
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Figure  4.  Vehicle  coordinate  system  (z-axis  is  down) 

2.2.2  Longitudinal  Speed 

The  longitudinal  speed  is  the  vehicle's  speed  along  the  x-axis.  The  ROR  report 
recommends  that  the  warning  system  should  determine  vehicle  velocity  to  an  accuracy  of 
3 miles  per  hour  (mph)  (L-20  page  24  in  [2]).  Use  the  time  to  travel  a known  distance  to 
verify  the  speedometer’s  accuracy. 

The  vehicle’s  speedometer  provides  vehicle  speed  during  tests.  An  independent 
measurement  system  (IMS)  verifies  the  vehicle’s  speed  during  a test  using  a dash  camera 
aimed  at  the  speedometer. 

2.2.3  Lateral  Position 

The  lateral  position  is  the  position  of  the  vehicle's  origin  relative  to  the  center  of  the  lane. 
The  lateral  position  measurement  is  along  the  vehicle  y-axis.  Most  measurements  in  this 
test  use  a similar  lateral  measurement  called  Distance  to  Road  Edge  (see  section  2.2.7). 

2.2.4  Lateral  Velocity 

The  lateral  velocity  is  the  rate  of  change  of  lateral  position. 

2.2.5  Yaw  Rate 

Yaw  rate  is  the  rate  of  rotation  about  the  vehicle  z-axis. 

2.2.6  Road  Departure 

The  vehicle  departs  the  road  when  the  outside  edge  of  any  of  the  vehicle  tires  crosses  the 
outside  edge  of  the  road. 

2.2.7  Distance  to  Road  Edge 

The  distance  to  the  road  edge  is  the  distance  from  the  vehicle's  outside  front  tire  to  the 
road  edge  measured  along  the  vehicle’s  y-axis. 
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Figure  5.  Distance  to  road  edge  is  perpendicular  to  vehicle 

The  tests  measure  the  lateral  position  using  the  IMS  (see  section  4)  video.  A calibration 
procedure  transforms  pixels  in  the  image  into  distance  to  road-edge  measurements.  The 
analyst  selects  the  pixel  on  the  road  edge  and  the  IMS  returns  the  distance. 

2.2.8  Road  Departure  Rate 

The  road  departure  rate  is  the  rate  of  change  in  distance  to  the  road  edge. 


Figure  6.  Computing  lateral  velocity 


Use  the  following  equation  to  compute  road  departure  rate: 


1 — Road  Departure  Rate 

tx  — tQ 

Where: 

yj,  yo  - are  the  lateral  positions  at  time  tj  and  to  respectively  (see  figure  above). 

Many  tests  require  the  driver  to  depart  the  road  at  a given  rate,  for  example,  above  0.75 
m/s.  The  driver  achieves  the  desired  departure  rate  by  traversing  a known  distance  in  a 
known  time  interval.  For  example,  the  driver  starts  the  departure  at  0.75  m from  the  lane 
marker  and  crosses  the  lane  marker  in  one  second.  Afterward,  the  analyst  uses  the  IMS 
calibrated  video  to  measure  the  exact  departure  rate. 
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2.2.9  Road  Departure  Angle 

The  road  departure  angle  is  the  angle  between  the  forward  x-axis  of  the  vehicle  and  the 
road  edge. 


Figure  7.  Departure  angle  on  straight  road.  V is  the  forward  velocity  and  is  the  lateral  velocity. 

The  following  formula  estimates  the  departure  angle  from  the  departure  rate  and  the 
forward  velocity: 


6°  = tan  1 


\ v A K 


Departure  Angle  (in  degrees)  from  forward  and 
lateral  velocities. 


Where: 

via  = is  the  road  departure  rate  (m/s)  (see  section  2.2.8) 
v = vehicle  forward  speed  (m/s) 

2.2.10  Distance  to  Curve  Entry  Point 

The  curve  entry  point  marks  the  transition  between  a straight  road  segment  and  a curved 
road  segment  (see  Figure  8).  The  distance  to  the  curve  entry  point  is  measured  using  the 
IMS  in  the  following  manner: 

Determine  the  start  location  of  the  measurement,  for  example,  the  time  of  a warning. 

Note  the  lateral  distance  to  the  departure-side  road  edge.  Record  the  initial  («2,  ei) 
location  of  the  vehicle. 

Advance  the  video  until  the  lateral  distance  visibly  decreases.  This  marks  the  curve  entry 
point.  Record  the  location  of  the  vehicle  at  the  curve  entry  point  (n2,  ei). 

Determine  the  distance  to  the  curve  entry  point  using  the  following  formula: 
xm  = -yji/i]  — )2  + ( ex  — e-,  )2  Measured  distance  to  Curve  Entry  Point 


12 


n 


Figure  8.  Measured  distance  to  curve  entry  point  at  time  of  warning 


2.3  Warning  System  Definitions 
2.3.1  System  Response  Rating 

The  rating  of  a system’s  response  to  a warning  situation  falls  into  one  of  the  following 
categories  [6]: 


System  Response 

Situations  Requiring  a 
Warning 

Situations  in  Which  a 
Warning  is  not  Required 

Warning 

True  Positive  (TP) 
Correctly  warns  of  road 
departure 

False  Positive  (FP) 

False  alarm 

No  Warning 

False  Negative  (FN) 
Misses  road  departure 

True  Negative  (TN) 
Correctly  does  not  warn  of 
road  departure. 

Table  2 Rating  of  warning  system  signals. 
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2.3.2  Imminent  Crash  Condition 

An  imminent  crash  condition  exists  when  there  is  a high  probability  of  a lateral  collision, 
a loss  of  steering  control  or  a rollover.  A RDCWS  considers  leaving  the  paved  portion  of 
the  road  at  a high  speed  as  an  imminent  crash  condition  because  of  the  high  probability  of 
loss  of  steering  control  and  rollover. 

2.3.3  Crash  Warning 

A crash  warning  is  a signal  to  the  driver  indicating  an  imminent  crash  condition  exists 
and  that  the  driver  should  take  immediate  action  to  prevent  the  crash. 

2.3.4  Advisory  Warning 

An  advisory  warning  is  a signal  to  the  driver  that  the  vehicle  is  about  to  drift  outside  the 
current  lane  or  that  the  current  speed  is  too  great  for  an  upcoming  curve. 

2.3.5  Warning  Sensitivity 

Warning  sensitivity  is  a user-controlled  variable  that  determines  the  timing  of  a warning. 
High  sensitivity  produces  earlier  warnings  and  low  sensitivities  produce  later  warnings. 
The  sensitivities  affect  evaluation  of  a warning  system  by  setting  a maximum  value  for 
lateral  acceleration  (due  to  driver  steering  correction)  on  a lateral  drift  departure  and 
required  longitudinal  deceleration  (due  to  driver  braking  to  slow  the  vehicle)  on  a curve 
over-speed  departure.  A summary  of  the  warning  system  sensitivities  appears  in  the 
following  table  [7].  Unless  stated  otherwise,  the  sensitivity  is  at  the  mid-range  value  (3). 


Sensitivity 

dlat 

m/s2 

dreq 

m/s" 

1 

4.12(0.42  g) 

6.86  (0.70  g) 

2 

3.53  (0.36  g) 

5.88  (0.60  g) 

3 

2.94  (0.30  g) 

4.90  (0.50  g) 

4 

2.35  (0.24  g) 

3.92  (0.40  g) 

5 

1.76  (0.18  g) 

2.94  (0.30  g) 

Table  3 Warning  system  sensitivity  values  for  lateral 
acceleration  (ato)  and  longitudinal  deceleration  ( dreq ), 
where  g equals  gravity  (9.8  m/s2). 


2.3.6  Reaction  Time 

The  reaction  time  defines  the  amount  of  delay  between  onset  of  a warning  and  onset  of  a 
steering  maneuver  by  the  driver.  An  average  reaction  time  for  these  tests  is  tr  = 1 .5  s.  A 
lower  bound  (//)  and  an  upper  ( th ) define  the  range  of  acceptable  reaction  times.  The 
evaluation  equations  use  these  values  to  determine  the  earliest  and  latest  location  of  a 
warning.  Table  4 gives  suggested  values  for  the  reaction  times. 
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Name 

Time 

Description 

U 

0.75  s 

Lower  bound  used  to  define  latest  possible 
warning  location. 

tr 

1.5  s 

Ideal  reaction  time 

th 

2.0  s 

Upper  bound  used  to  define  earliest 
possible  warning  location. 

Table  4.  Reaction  time  with  low  and  high  limits. 


2.3.7  Obstacle 

An  obstacle  is  an  object  of  sufficient  mass  to  cause  damage  to  a vehicle  upon  collision. 


2.3.8  Radar  Corner  Cube  Reflector 

Radar  cross  section  (RCS)  is  the  measure  of  a target's  ability  to  reflect  radar  signals  in  the 
direction  of  the  radar  receiver.  The  comer  reflector  has  an  RCS  almost  as  high  as  a flat 
plate  but  over  a wider  reflectance  angle  (over  ±60°). 


Figure  9.  Corner  Cube  Reflector 


Use  the  following  equation  to  determine  the  height  of  a comer  cube  for  a desired  RCS. 


H = 


3A2g 


Height  of  corner  cube 


{ 4x  ) 

Where: 

H = height  of  comer  cube  (m) 
a = desired  radar  cross  section  (RCS)  (m  ) 
A = radar  wavelength  (m) 


2.3.9  Time  of  Warning 

The  time  of  warning  is  the  time  when  the  warning  system  issues  an  alert.  Determining 
whether  a warning  is  late,  early  or  on  time  requires  the  time  of  the  warning  relative  to  the 
road  departure. 


The  IMS  (see  section  4)  captures  the  time  of  the  warning  from  the  visual  icon  on  the 
warning  system  display  or  from  the  audible  alert  (using  a microphone).  The  analyst 
searches  through  the  video  to  locate  the  warning  and  uses  the  GPS  clock  and  the  video 
frame  clock  (the  frames  are  29.9  ms  apart)  to  measure  the  time  of  the  warning. 
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2.3.10  Time  to  Line  Crossing 

The  time  to  line  crossing  is  the  calculated  time  until  the  vehicle's  lead  tire  crosses  an  in- 
path lane  marker  given  the  vehicle  continues  on  its  current  trajectory. 

2.3.11  Time  to  Road  Departure 

The  time  to  departure  is  the  time  until  the  vehicle's  lead  tire  crosses  a line  parallel  to  the 
outer  lane  marker  offset  by  the  available  maneuver  room. 

2.3.12  Nuisance  Alarm 

Nuisance  alarms  are  warnings  deemed  unwarranted  by  a driver.  They  include  false 
alarms  due  to  incorrectly  predicting  the  time  or  position  of  a crash  (i.e.,  early  warning) 
and  false  positives  due  to  incorrect  sensor  readings  (from  noisy  or  cluttered 
environments). 

2.4  Environmental  Definitions 

2.4.1  Daylight  Time 

Daylight  is  the  times  of  day  starting  at  sunrise  and  ending  at  sunset.  The  times  are 
available  from  the  US  Naval  Observatory  (http  ://aa.u  sno  .na  vy . mi  1/data  A . 

2.4.2  Night  Time 

Night  is  the  times  of  day  after  civil  twilight  at  sunset  and  ending  at  twilight  before 
sunrise.  Civil  twilight  is  when  the  sun  is  6 degrees  below  the  horizon.  According  to 
definitions  used  by  the  US  Naval  Observatory: 

In  the  morning  before  the  beginning  of  civil  twilight  and  in  the  evening  after  the  end 
of  civil  twilight,  artificial  illumination  is  normally  required  to  carry  on  ordinary 
outdoor  activities.  Complete  darkness,  however,  ends  sometime  prior  to  the 
beginning  of  morning  civil  twilight  and  begins  sometime  after  the  end  of  evening 
civil  twilight. 

The  times  of  civil  twilight  are  available  from  the  US  Naval  Observatory 

(http://aa.usno.naw.mil/dataA. 

2.4.3  Rain 

Rain  is  precipitation  in  the  form  of  water  drops  larger  than  0.5  mm  (0.02  in).  The 
following  table  shows  how  rainfall  intensity  is  determined: 


Rainfall 

Intensity 

Rate-of-fall  in  6 min 

Visual  Estimation 

Light 

< 0.25  mm 
(0.01  in) 

From  scattered  drops  that,  regardless  of 
duration,  do  not  completely  wet  an  exposed 
surface  up  to  a condition  where  individual 
drops  are  easily  seen 

Moderate 

0.25  mm  to 

From  scattered  drops  that,  regardless  of 
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0.762  mm  (0.03  in) 

duration,  do  not  completely  wet  an  exposed 
surface  up  to  a condition  where  individual 
drops  are  easily  seen. 

Fleavy 

> 0.762  mm 

Rain  seemingly  falls  in  sheets;  individual  drops 
are  not  identifiable;  heavy  spray  to  the  height 
of  several  centimeters  over  hard  surfaces. 

Table  5.  Rain  rate  of  fall. 


2.4.4  Fog 

Fog  is  as  a visible  aggregate  of  minute  water  particles  (droplets),  which  is  close  to  the 
Earth's  surface  and  reduces  horizontal  visibility  to  less  than  1 km  (5/8  mi),  and  unlike 
drizzle,  it  does  not  fall  to  the  ground. 

Names  given  to  fog  types  identify  their  methods  of  formation.  The  principle  types  are 
radiational  fog,  ice  fog,  advection  fog,  upslope  fog,  rain-induced  fog,  and  steam  fog. 
These  types  of  fog  are  called  "dense"  when  the  surface  visibility  is  equal  to  or  less  than 
400  m (1/4  mi).  A Dense  Fog  Advisory  occurs  when  the  dense  fog  becomes  widespread. 

2.4.5  Snow 

Snow  precipitation  consists  of  water  crystals,  mostly  branched  in  the  form  of  six-pointed 
stars.  It  usually  falls  steadily  for  several  hours  or  more.  Qualifiers,  such  as  occasional  or 
intermittent,  indicate  a steady,  prolonged  (for  several  hours  or  more)  fall  is  not  expected. 
Like  rain,  visibility  determines  snow  intensity.  The  following  table  shows  snow  intensity 
versus  visibility: 


Snow  Intensity 

Visibility 

Light 

> 1 km  (5/8  mi) 

Moderate 

400  m ( 1/4  mi)  to  1 km 

Heavy 

<400  m 
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3 Basic  Test  Organization 

Each  test  procedure  in  this  document  consists  of  the  following  subsections. 

Test  Title 

Each  test  in  this  document  starts  with  a short  description  of  the  purpose  of  the  test  and  a 
figure  (see  Figure  10)  showing  the  trajectory  of  the  vehicle,  the  road  geometry,  pertinent 
test  props  and  key  events  of  the  test. 


Scenario  variables 


Vehicle  trajectory  Barrier 

Figure  10.  Each  test  procedure  includes  a figure  that  illustrates  pertinent  aspects  of  the  test. 
Events  and  System  Response 

This  section  describes  the  events  during  the  test  and  the  response  of  the  warning  system 
to  the  event.  Typically,  the  response  is  a specific  warning  signal. 
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Conditions 

This  section  describes  the  conditions  for  the  test.  Some  conditions  remain  fixed 
throughout  the  test,  for  example,  road  geometry  and  lane  marker  quality.  Other 
conditions  may  vary  between  a set  of  tests,  for  example,  speed  and  departure  rate. 

Measurements 

This  section  lists  all  the  measurements  that  are  required  for  the  tests.  Also  presented  are 
methods  and  procedures  for  processing  the  measurements. 

Procedure 

This  section  lists  the  steps  of  the  test,  which  have  the  following  format: 

1 . Setup  and  verify  scenario  conditions. 

2.  Drive  system/vehicle  along  desired  trajectory 

3.  Record  trajectory  and  warning  system  responses 

4.  Process  data  to  determine  performance  variables 

5.  Evaluate  the  system  response  to  determine  whether  system  passed 

Evaluation  Methods 

This  section  contains  the  methods  forjudging  whether  the  system  passes  the  test.  The 
methods  used  for  most  tests  are: 

1.  Is  the  correct  warning  system  response  given  (e.g.,  did  the  system  miss  the  event)? 

2.  Is  the  timing  of  the  warning  acceptable? 
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4 Independent  Measurement  System 

This  section  describes  an  independent  measurement  system  used  to  collect  warning- 
system  performance  data  required  for  system  evaluation.  Evaluation  of  the  warning 
system  should  use  sensors  and  equipment  separate  from  the  warning  system  under  test. 
For  example,  the  distance  to  a lane  marker  at  the  time  of  warning  should  not  come  from 
the  warning  system  itself.  Errors  in  the  warning  system  camera  and  image-processing 
algorithm  could  lead  to  incorrect  assessment  of  warning  system  performance.  Details  of 
the  IMS  appear  in  [5].  This  section  provides  enough  details  to  understand  one  approach 
for  collecting  data  and  measurements  to  analyze  warning  system  performance. 


DGPS  Receiver/Modem 


analog 

video 


DV 

Recorder 


forward  and  side  cameras 


Multiplexer 


audio 

channel 


dash  camera 


cab  microphone 


Figure  11.  Independent  Measurement  System 


Figure  1 1 illustrates  the  components  of  the  IMS  described  [5].  Three  cameras  situated 
on  a roof  rack  provide  forward  and  side-looking  video.  A fourth  camera  mounted  on  the 
dash  captures  the  warning  system  display.  A quad  multiplexer  combines  the  four  video 
signals  into  a single  video  stream.  The  output  of  the  quad  appears  in  Figure  12. 
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Figure  12.  Four  camera  views  combined  into  one.  Top  left  and  right  are  view  of 
calibration  sticks  aligned  with  front  wheels.  Bottom  right  shows  calibration 
sticks  on  the  road  in  front  of  vehicle.  Bottom  left  is  view  of  warning  system 
display. 


The  cameras  perform  the  following  functions: 

1 . The  left  and  right  downward  facing  cameras  measure  lateral  distances  relative  to  the 
front  wheels.  The  top  left  and  right  quad  in  Figure  12  show  calibration  sticks  for 
calibrating  the  side  looking  cameras.  The  black  and  white  stripes  on  the  calibration 
sticks  have  known  dimensions.  The  user  identifies  the  pixel  locations  of  the  stripes 
(see  red  circles  in  top  left  of  Figure  13,  which  defines  a transformation  between  pixel 
coordinates  in  the  image  and  ground  coordinates  relative  to  the  front  wheels.  After 
camera  calibration,  the  sticks  are  no  longer  required.  The  overlaid  red  circles  indicate 
the  calibrated  portion  of  the  video  data  (see  top  right  quad  in  Figure  13).  Clicking  on 
a pixel  centered  with  the  red  circle  returns  the  lateral  distance  as  if  the  calibrated 
sticks  existed.  Linear  interpolation  gives  distance  measurements  between  the  circles. 

2.  A forward  facing  camera  provides  lateral  measurements  forward  of  the  vehicle.  The 
camera  calibration  uses  calibration  sticks  as  well.  This  view  provides  a better  view  of 
the  surrounding  environment  then  just  the  side  looking  cameras. 

3.  A dash  view  camera  shows  vehicle  speed  and  when  an  alert  is  given. 
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Figure  13.  Result  of  pixel  to  ground  calibration  is  shown  using  red  circles. 
Circles  in  top  left  view  identify  transitions  from  black  to  white  marking  on 
calibration  stick.  After  removal  of  the  calibration  stick  (e.g.,  top  right 
view),  the  circles  provide  a reference  for  the  operator  when  measuring 
lateral  distances  from  the  front  wheel. 


Other  components  of  the  IMS  include  a Differential  Global  Positioning  Satellite  (DGPS) 
receiver  to  sense  vehicle  position  and  velocity,  and  a cab  microphone  to  pick  up  the 
audible  alerts  and  comments  from  the  driver.  To  simplify  data  collection,  a modem-like 
device  converts  the  GPS  data  into  an  audible  signal.  A digital  videocassette  recorder 
(DV  VCR)  simultaneously  records  the  output  of  the  video,  microphone  (one  audio 
channel)  and  GPS  (the  second  audio  channel),  ensuring  synchronization  of  the  data 
streams. 
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5 Warning  System  Evaluations 

This  section  describes  various  methods  for  rating  warning  system  performance  that  are 
common  among  tests.  The  methods  describe  how  to  score  the  system,  but  the  methods 
do  not  specify  what  score  is  acceptable.  Some  example  criteria  for  establishing  an 
acceptable  score  include: 

1.  The  system  must  pass  95%  of  the  runs  for  a given  test  (good  when  there  is  large 
number  of  test  runs) 

2.  The  system  may  only  fail  1 time  for  a given  set  of  test  runs  (good  when  there  is  a 
small  number  of  test  runs) 

The  organization  interested  in  the  test  results  should  specify  the  acceptable  pass/fail 
criterion. 

5.1  Lateral  Drift  Evaluation 

A warning  must  provide  a driver  with  time  to  react  and  time  to  avoid  (i.e.,  steer  away 
from)  a departure.  The  reaction  phase  starts  at  the  time  of  warning  and  continues  until 
the  driver  initiates  the  steering  correction.  The  steer  phase  consists  of  the  trajectory  the 
vehicle  follows  during  the  steering  correction.  The  magnitude  of  the  steer  should  not 
exceed  aiat  limits  defined  in  Table  3 of  section  2.3.5  (Warning  Sensitivity).  The  lateral 
drift  analysis  uses  separate  equations  for  straight  or  curved  roads. 

5.1.1  Straight  Road 

A lateral  drift  on  a straight  road  occurs  when  the  vehicle  departs  a straight  road  at  a fixed 
angle  to  the  direction  of  the  roadway. 


Figure  14.  Geometry  for  a lateral  drift  scenario  on  a straight  road 

Given  the  geometry  of  the  lateral  departure  of  a straight  road  shown  in  Figure  14,  one 
determines  the  desired  lateral  distance  to  issue  an  alert  using  the  following  equation: 
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vtr  sin  6 + r(l  - cos  0 ) 


Where: 

v = vehicle  forward  speed  (m/s) 

tr  = delay  between  warning  and  start  of  steering  (see  section  2.3.6)  (s). 

0 = the  departure  angle  (radians).  If  measuring  departure  rate,  convert  to  angle  using 

equation  in  section  2.2.9). 

r = the  radius  of  the  curve  the  vehicle  must  travel  to  avoid  the  departure  (m) 

The  equation  simplifies  to: 
yw  = vtr  tan  6 + r(  1/ cos  6 — l) . 


Use  the  measured  lateral  velocity  (viat)  to  calculate  the  departure  angle  (see  section  2.2.9): 
6 = tan  1 


r v,  ^ 

lat 


V v ) 


The  radius  should  not  cause  the  vehicle  to  exceed  a maximum  lateral  acceleration  (aiat, 

see  section  2.3.5),  therefore: 

2 

V 

r = 

aiat 

To  determine  the  desired  location  of  a warning  (the  lateral  distance  to  road  boundary)  for 
comparison  with  the  measured  time  of  a warning: 

1 . Measure  v and  v/at  at  the  time  of  warning. 

2.  Obtain  tr  from  section  2.3.6  and  a/at  from  section  2.3.5  sensitivity  setting  3. 

3.  Calculate  desired  location  using  the  following  equation: 


y.=VlJr+  — {i/COS0- 0 

a,„t 


(1) 


Use  the  following  procedure  to  determine  whether  the  actual  measured  warning  location 
during  a test  is  early , late  or  on  time. 

1 . Use  the  desired  location  warning  equation  to  determine  the  latest  warning  location 
(L  WL)  (warning  occurs  closer  to  road  boundary)  using  sensitivity  setting  1 (highest 
a/at  from  section  2.3.5)  and  reaction  time  t/  (shortest  reaction  time  from  section  2.3.6). 

2.  Use  the  desired  location  warning  equation  to  determine  the  earliest  warning  location 
( EWL ) (warning  occurs  farther  from  road  boundary)  using  sensitivity  setting  5 
(lowest  aiat  from  section  2.3.5)  and  reaction  time  th  (longest  reaction  time  from 
section  2.3.6).  Use  the  following  relationships  to  compare  the  measured  location  of 
the  warning  (ym ) with  LWL  and  EWL  to  determine  whether  the  warning  is  early , late 
or  on  time  (refer  to  Figure  16): 

If ym>  EWL,  then  the  warning  is  early 

if 'ym  < L WL , then  the  warning  is  late 

If  EWL  >ym<L  WL , then  the  warning  is  on  time 
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Figure  15  Relationship  to  determine  timeliness  of  warning  on 
lateral-distance  based  warning. 


5.1.2  Curved  Road 

A lateral  drift  on  a curved  road  occurs  when  the  vehicle  is  traveling  straight  and  departs  a 
curved  road. 


Figure  16.  Geometry  for  a lateral  drift  scenario  on  a curved  road 

Reference  [10]  describes  the  derivation  of  the  lateral  acceleration  required  to  avoid  a 
lateral  departure  on  a curved  road  (see  Figure  16  - note  the  change  in  x and  y axes  to 
remain  consistent  with  y measurements  being  lateral  from  the  vehicle): 
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Where: 

v = vehicle  forward  speed  (m/s) 
rc  = the  radius  of  the  curve  (m) 

tr  = delay  required  by  driver  to  initiate  the  steering  correction 
yw  = the  lateral  distance  to  the  center  of  the  curve  (m). 

xw  = the  longitudinal  distance  to  the  curve  entry  point  (m)  (See  section  2.2.10.) 
Subject  to: 


(3) 


(The  onset  of  steering  must  occur  prior  to  the  road  boundary). 

Note  that  in  this  scenario,  the  vehicle  must  travel  straight  during  the  departure  (i.e., 
remain  parallel  to  the  road  edge  until  entering  the  curve).  The  initial  lateral  position  (yw) 
dictates  the  ideal  location  for  a warning  (xWt),  which  is  relative  to  the  curve  entry  point. 
The  curve  entry  point  is  the  point  where  the  lateral  distance  to  the  road  edge  starts  to 
decrease.  Using  the  IMS,  the  distance  is  derived  from  speed  and  time. 

To  determine  the  desired  location  of  a warning  (longitudinal  distance  to  curve  entry)  for 
comparison  with  the  measured  time  of  a warning: 

1 . Measure  v,  the  lateral  distance  to  the  road  edge  ym,  and  rc  during  a test. 

2.  Determine  yw  from  yw  = rc-ym. 

3.  Obtain  tr  from  section  2.3.6  and  aiat  from  section  2.3.5  sensitivity  setting  3. 

4.  Calculate  desired  location  using  the  following  equation: 


(4) 


Use  the  following  procedure  to  determine  whether  the  actual  measured  warning  location 

during  a test  is  early , late  or  on  time. 

1 . Measure  v,  the  lateral  distance  to  the  road  edge  ym,  the  longitudinal  distance  to  the 
curv  e entry  xm  (note:  xm  is  negative  if  vehicle  hasn’t  entered  the  curve)  and  rc  at  the 
time  of  a warning. 

2.  Determine  yw  and  xw  from  yw  = rc—  ym  and  xw  = xm. 

3.  If  vtr  >yjrc2  -yl  -xw  then  turn  begins  after  vehicle  leaves  the  lane  (can’t  use  equation 
(2)).  Classify  warning  as  late. 

4.  Use  equation  (2)  to  determine  the  lateral  acceleration  required  to  stay  on  the  road. 

5.  If  aiat  > sensitivity  setting  1 (highest  aiat  from  section  2.3.5)  then  classify  warning  as 
late. 

6.  If  aiat  < sensitivity  setting  5 (lowest  aiat  from  section  2.3.5)  then  classify  warning  as 


early. 


7.  Else  aiat  falls  within  the  acceptable  limits  and  classify  the  warning  as  on  time. 

5.2  Curve  Speed  Evaluation 

A curve-speed  warning  must  provide  a driver  with  time  to  react  (reaction  phase)  and  to 
slow  (brake  phase)  to  a safe  speed.  The  braking  action  should  not  exceed  a limit  on 
deceleration. 


Figure  17.  Geometry  for  a curve  speed  warning  scenario. 


Use  the  following  equation  to  calculate  the  amount  of  deceleration  required  to  reach  the 
safe  speed: 


2 2 

d = — — — 


(5) 


Where: 

dreq  = required  deceleration  (m/s  ) 
v = vehicle  initial  forward  speed  (m/s) 
v5  = safe  speed  for  the  curve  (m/s) 
tr  = driver  reaction  time  (s) 

xw  = the  distance  between  the  warning  location  and  the  start  of  the  curve  (m)  (See 
section  2.2.10.) 


The  safe  speed  for  a curve  of  radius  r does  not  exceed  a maximum  lateral  acceleration 
(aiah  see  section  2.3.5).  Use  the  following  equation  to  calculate  the  safe  speed: 

vs  = V7T  (6) 

To  determine  the  desired  location  of  a warning  (distance  to  curve  entry)  for  comparison 
with  the  measured  time  of  a warning: 

1 . Measure  v,  ya,  and  rc  during  a test. 

2.  Obtain  tr  from  section  2.3.6,  and  aiat  and  dreq  from  section  2.3.5  sensitivity  setting  3. 

3.  Calculate  desired  location  using  the  following  equation: 


x 


vv 


2d 


req 


+ tv 


(7) 


Use  the  following  procedure  to  determine  whether  the  actual  measured  warning  location 
during  a test  is  early , late  or  on  time. 
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1.  Use  the  desired  location  warning  equation  to  determine  the  latest  warning  location 
(L  WL)  (warning  occurs  closer  to  road  boundary)  using  sensitivity  setting  1 (highest 
aiat  from  section  2.3.5)  and  reaction  time  tj  (shortest  reaction  time  from  section  2.3.6). 

2.  Use  the  desired  location  warning  equation  to  determine  the  earliest  warning  location 
(EWL)  (warning  occurs  farther  from  road  boundary)  using  sensitivity  setting  5 
(lowest  aiat  from  section  2.3.5)  and  reaction  time  th  (longest  reaction  time  from 
section  2.3.6). 

3.  Use  the  following  relationships  to  compare  the  measured  location  of  the  warning  (xm) 
with  LWL  and  EWL  to  determine  whether  the  warning  is  early , late  or  on  time  (refer 
to  Figure  18): 


\fxm  > EWL , then  the  warning  is  early 

If  xm  < L WL , then  the  warning  is  late 

If  EWL  > xm  < LWL , then  the  warning  is  on  time 


Figure  18.  Relationship  to  determine  timeliness  of  warning  on  lateral-distance  based  warning. 


5.3  Timeliness  of  Warning 

Once  warnings  are  evaluated  as  early , late  or  on  time  for  the  given  scenario  (departure  on 
straight  road,  departure  on  curved  road,  curve  speed  departure),  the  results  are 
accumulated  as  follows: 


r # Early  s 

k #TP  , 


•100 


r #OnTime\ 
^ #TP  J 


r #Late^ 

C#TP; 


•100 


Percent  of  Warnings  Early 


Percent  of  Warnings  on  Time 


Percent  of  Warnings  Late 
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Where  tiEarly,  ttOnTime,  #Late,  and  #TP  are  the  total  number  of  early  warnings,  on  time 
warnings,  late  warnings,  and  the  true  positive  warnings.  Each  test  procedure  describes 
how  to  determine  if  a warning  is  a true  positive  (also  referred  to  as  the  correct  response 
(CR)).  The  results  of  several  test  runs  are  summarized  in  the  following  table. 


Test  Run  Summary  Results 


Combined  Results  for 
all  runs 

#of 

runs 

# 

CR 

# 

OT 

# 

E 

# 

L 

% 

OT 

% 

E 

% 

L 

Date/time  of  test  runs 

5.4  Efficacy  Rate 

Reference  [6]  defines  the  efficacy  rate  as  the  rate  that  a desired  effect  occurs. 


efficacy  rate 


' #TP  ' 
k#TP+#FNj 


•100 


5.5  False  Alarm  Rate 

Reference  [6]  defines  the  false  alarm  rate. 


false  alarm  rate  = 


#FP  ' 
#TP+#FP j 


•100 
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6 Tests  for  Departure  on  Roads  with  Default  Minimum  AMR 

The  tests  in  this  section  evaluate  warning  system  response  to  road  departures  where  the 
available  maneuver  room  is  at  a default  minimum.  This  situation  arises  when  the  sensor 
processing  has  unacceptable  uncertainty,  when  maps  of  the  road  network  are  unavailable, 
or  when  the  warning  system  lacks  the  sensing  capability  to  detect  the  drivable  surface  on 
the  side  of  the  road. 

The  tests  evaluate  the  system’s  ability  to: 

1 . Detect  lane  markers, 

2.  Determine  lane  marker  type, 

3.  Determine  departure  velocities, 

4.  Determine  driver  lane  change  intent 

The  test  scenarios  include  departures  on  a divided  straight  road,  on  an  undivided  straight 
road  and  on  a divided  curved  road. 


6.1  One-Way  Straight  Road  (Default  Minimum  AMR) 

This  test  assesses  the  performance  of  the  lateral  drift  warning  function  for  road  departures 
on  divided  straight  roads.  This  scenario  falls  under  the  "going  straight  on  freeway  with 
two  lanes  at  non-junction"  scenario  identified  in  the  NHTSA  test  scenario  report  as  one 
of  the  major  settings  for  road  departure  crashes  [3].  The  test  also  examines  warning 
system  response  during  lane  changes  (i.e.,  when  vehicle  crosses  dashed  lane  markers) 
with  and  without  turn  signals. 

The  figure  below  illustrates  the  test  scenario.  Both  a left  and  a right  departure  are  tested. 


A B 

Figure  19.  Scenario  for  road  departure  on  divided  road 

6.1.1  Events  and  System  Response 

The  following  table  defines  the  events  for  this  test  and  the  correct  warning  signal  the 
system  shall  provide.  See  the  figure  above  to  locate  the  event  (e.g.,  Id  A1  is  label  1 in 
figure  A). 
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Id 

Event 

Correct  Response 

A1 

Crossed  dashed  line 

1 . Issue  no  warning  if  correct  turn  signal  active. 

2.  Issue  an  advisory  warning  if  no  turn  signal  active 
or  incorrect  turn  signal  active. 

A2 

Road  departure 

1 . Issue  road  departure  warning 

B1 

Same  as  A1 

1.  Same  as  A 1 

B2 

Same  as  A2 

1.  Same  as  A2 

6.1.2  Test  Conditions 

The  conditions  for  this  test  include  the  following. 

6.1.2. 1 Road  Geometry 

The  road  geometry  for  the  test  is  the  following: 


Road  Geometry 

Dimension 

Straight  (defined  using  radius  of  curvature) 

> 1000  m 

Appendix  A,  Methods  for  Determining  Road  Dimensions  and  Geometry,  contains  the 
recommended  method  to  verify  the  road  geometry. 

6.1.2.2  Available  Maneuver  Room 

In  this  test,  the  warning  system  recognizes  the  road  edge  but  cannot  determine  if  there  is 
additional  AMR.  Therefore,  the  system  should  assume  a default  minimum  AMR. 


Available  Maneuver  Room 

Comment 

15  cm  (6  in) 

Default  minimum  AMR 

6.1.2.3  Speed 

The  vehicle  speeds  during  the  tests  cover  warning  system  operating  range. 


Low  Speed 

Medium  Speed 

High  Speed 

64  km/h  (40  mph) 

89  km/h  (55  mph) 

1 13  km/h  (70  mph) 

6. 1.2.4  Departure  Rate 

The  vehicle  departure  rates  are  either  high  or  low.  These  rates  are  comparable  to  the 
rates  suggested  in  the  ROR  report  and  in  the  ISO  standard  for  testing  lateral  drift  warning 
systems. 


Low  Departure  Rate 

High  Departure  Rate 

< 0.75  m/s 

> 0.75  m/s 

6.1.3  Measurements 

Measurements  collected  during  the  test  support  the  following: 
1 . Determine  whether  a warning  was  given 
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2.  Determine  the  desired  location  of  a warning  given  the  conditions 

3.  Determine  the  amount  of  time  the  warning  gives  driver 

6.1.3. 1 Time  of  Warning 

See  section  2.3.9. 

6.1.3.2  Longitudinal  Speed 

See  section  2.2.2. 

6.1.3.3  Distance  to  Road  Edge 

See  section  2.2.7. 

6.1.3.4  Lateral  Velocity 

See  section  2.2.4. 

6.1.4  Test  Procedure 

The  general  procedure  for  the  test  repeats  a series  of  road  departure  trajectories  depicted 

in  Figure  19. 

6.1.4. 1 Pre-Test  Procedures 

The  procedures  before  the  test  are: 

1 . Ensure  road  geometry  meets  the  test  condition.  Measure  the  geometry  if  it  is 
unknown  (see  Appendix  A,  Methods  for  Determining  Road  Dimensions  and 
Geometry). 

2.  Ensure  lane  marker  characteristics  meet  the  test  conditions  (see  sections  2.1.1- 
2.1.12). 

3.  Check  speedometer  accuracy  (see  section  2.2.2). 

4.  Verify  IMS  camera  calibration  according  to  recommendations  in  [5].  Repeat 
camera  calibration  if  camera  has  moved. 

5.  Ensure  microphone  detects  warning  system  alerts  and  driver  speech. 

6.  Ensure  recording  system  is  working  properly. 

7.  Ensure  GPS  has  sufficient  satellite  signals. 

8.  Verify  road  boundary  for  test  (i.e.,  outer  lane  marker  plus  AMR)  and  practice 
vehicle  trajectories  (speeds,  departure  rate,  etc.)  until  accomplished  safely  and 
reliably. 

6.1.4.2  Test  Procedures 

The  procedures  during  the  test  are: 

1 . Achieve  speed.  The  driver  brings  the  vehicle  to  the  prescribed  speed  and  sets  the 
cruise  control  at  the  speed. 

2.  Set  turn  signal.  The  driver  runs  the  first  departure  with  turn  signal  active.  Since  this 
departure  will  be  a True  Negative,  the  operator  documents  the  departure  time.  The 
operator  runs  the  remaining  departures  without  activating  the  turn  signal. 
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3.  Execute  departure.  The  driver  steers  toward  the  road  boundary  at  the  prescribed 
departure  rate  (high  or  low)  until  the  road  boundary  is  crossed.  See  section  2.2.8  for 
a method  to  achieve  the  prescribed  departure  rate. 

4.  Record  departure  rate.  The  assistant  records  the  estimated  departure  rate  as  either 
low  or  high. 

5.  Go  to  start  of  next  departure  trajectory.  The  driver  steers  the  vehicle  back  onto  the 
road  for  the  next  departure. 

6.  The  driver  repeats  the  departure  trajectories  to  fulfill  the  following  conditions: 

a.  Five  left  departures  at  the  low  departure  rate 

b.  Five  right  departures  at  the  low  departure  rate 

c.  Five  left  departures  at  the  high  departure  rate 

d.  Five  right  departures  at  the  high  departure  rate 

e.  One  right  departure  at  the  high  departure  rate  with  the  right  turn  signal  on. 

f.  One  left  departure  at  the  high  departure  rate  with  the  left  turn  signal  on. 

7.  Repeat  steps  1 - 6 for  the  middle  prescribed  speed. 

8.  Repeat  steps  1 - 6 for  the  high-prescribed  speed. 

6.1.4.3  Post-Test  Procedures 

The  procedures  after  the  test  consist  of  the  following  steps: 

1.  Verify  IMS  camera  orientation  has  not  changed.  Repeat  tests  if  changed. 

6.1.5  Evaluation  Methods 

Use  the  following  methods  to  evaluate  warning  system  performance. 

6.1.5.1  Rate  System  Response 

Examine  the  video  and  audio  data  and  rate  each  warning  scenario. 

For  the  road  departure  events: 

The  rating  is  a true  positive  if  the  system  responds  with  a warning  before  the  road 
boundary  is  crossed. 

The  rating  is  a false  negative  if  the  system  does  not  respond  with  a warning  before 
the  road  boundary  is  crossed. 

For  the  cross  dashed-line  events: 

The  rating  is  a true  positive  if  the  system  responds  with  a warning  before  the  road 
boundary  is  crossed  and  the  turn  signal  is  inactive. 

The  rating  is  a false  positive  if  the  system  responds  with  a warning  before  the 
road  boundary  is  crossed  and  the  turn  signal  is  active. 
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6.1.5.2  Desired  Location  of  Warning 

For  each  test  run,  the  evaluator  determines  the  desired  location  for  the  warning.  The 
desired  location  depends  on  the  measured  departure  rate  and  the  forward  speed. 

1.  Locate  the  time  of  warning  in  the  video  data  (e.g.,  locate  first  video  frame  alert). 

2.  Measure  lateral  position  and  lateral  velocity  to  the  road  edge  at  time  of  warning. 

3.  Use  equation  (1)  in  section  5.1.1  to  determine  the  desired  location  for  a warning  on  a 
straight  road. 

6.1.5.3  Timeliness  of  Warning 

For  each  test  run,  the  evaluator  determines  whether  the  warning  is  early , late  or  on  time 
using  steps  described  in  section  5.1.1.  The  results  of  the  timeliness  evaluations  are 
accumulated  according  to  section  5.3. 

6.1.6  Reporting 

The  evaluator  compiles  the  test  reports  in  accordance  with  the  following  tables. 


Name  of  Test 

Departure  on  One-Way  Straight  Road 
with  Fixed  Available  Maneuver  Room 

Date  of  Test 

Warning  System  ID 

Name  and  Version  Number 
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Test  Run  Results 


Factors 

Response 

Speed 

Departure 

Direction 

Signal 

# of 

Run 

CR 

Vf 

Vi 

yw 

T 

mph 

Rate  m/s 

runs 

# 

T/F 

mph 

m/s 

m 

m 

OT/E/L 

40 

<0.75 

Left 

No 

5 

1 

If 

If 

II 

II 

2... 

40 

<0.75 

Right 

No 

5 

40 

>0.75 

Left 

No 

5 

40 

>0.75 

Right 

No 

5 

40 

>0.75 

Left 

On 

1 

55 

<0.75 

Left 

No 

5 

55 

<0.75 

Right 

No 

5 

55 

>0.75 

Left 

No 

5 

55 

>0.75 

Right 

No 

5 

55 

>0.75 

Left 

On 

1 

55 

>0.75 

Right 

On 

1 

70 

<0.75 

Left 

No 

5 

70 

<0.75 

Right 

No 

5 

70 

>0.75 

Left 

No 

5 

70 

>0.75 

Right 

No 

5 

70 

>0.75 

Left 

On 

1 

70 

>0.75 

Right 

On 

1 

CR  = System  gave  correct  response  (T)  or  failed  to  give  correct  response  (F).  Refer  to 
subsection  1 . 1 above  to  determine  correct  response. 

Vf  = measured  vehicle  forward  speed 

vi  = measured  vehicle  lateral  speed  (departure  rate) 

ym  = measured  lateral  distance  between  the  vehicle  front  tire  and  the  road  boundary  at 
the  time  of  warning 

yw  = the  desired  lateral  distance  at  the  time  of  warning  using  equation  (1) 

T = Timeliness  of  warning  OnTime,  Early  or  Late 


Test  Run  Summary  Results  (see  section  5.3) 


Combined  Results  for 
all  runs 

#of 

runs 

# 

CR 

# 

OT 

# 

E 

# 

L 

% 

OT 

% 

E 

% 

L 

Date/time  of  test  runs 

False  Alarm  Incident  Report 


Run 

# 

Probable  Cause 

1 

Example:  shadows  on  road  misinterpreted  as  road  edge 

Lane  Change  with  Turn  Signal  Event  Evaluation 


Number  of  Lane 
Change  Tests 

Number  of  Incorrect 
Responses 

Number  of  Correct 
Responses 

# 

# 

# 
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■ 
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6.2  Two-Way  Straight  Road  (Default  minimum  AMR) 

The  Two-Way  Straight  Road  (with  default  minimum  AMR)  test  assesses  the  performance 
of  the  lateral  drift  warning  function  for  road  departures  on  undivided  straight  roads.  This 
scenario  falls  under  the  "going  straight  on  undivided  non-freeway  with  two  lanes  at  non- 
junction" scenario  identified  in  the  NHTSA  test  scenario  report  as  one  of  the  major 
settings  for  road  departure  crashes  [3]. 

The  figure  below  illustrates  the  test.  The  road  boundary  depends  on  the  type  of  marker 
the  vehicle  first  encounters  during  the  departure  trajectory.  A solid  marker  prohibits 
passing.  The  road  boundary  is  the  left  edge  of  the  solid  marker  and  the  available 
maneuver  room  is  zero.  A dashed  marker  allows  passing  and  indicates  a wider  lane  but 
no  AMR.  The  warning  system  issues  a departure  alert  only  if  the  turn  signal  is  not  active. 
This  test  has  no  available  maneuver  room  because  the  adjacent  lane  may  have  undetected 
on-coming  traffic.  The  double  solid  lane  case  is  similar  to  the  solid; dashed  marker  in 
figure  A. 


A 


B 


Figure  20.  Vehicle  trajectory  during  test 


6.2.1  Events  and  System  Response 

The  following  table  defines  the  events  for  this  test  and  the  correct  warning  signal  the 
system  shall  provide.  See  the  figure  above  to  locate  the  event  (e.g.,  Id  A1  is  label  1 in 
figure  A). 
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Id 

Event 

Correct  Response 

A1 

Road  Departure  with 
passing  not  permitted 

1 . Issue  road  departure  warning 

B1 

Road  Departure  with 
passing  permitted 

1 . Issue  no  warning  if  the  correct  turn  signal  active. 

2.  Issue  a road  departure  warning  if  no  turn  signal 
active  or  incorrect  turn  signal  active. 

6.2.2  Test  Conditions 

The  conditions  for  this  test  include  the  following. 


6.2.2. 1 Road  Geometry 

The  road  geometry  for  the  test  is  the  following: 


Road  Geometry 

Dimension 

Straight  (defined  using  radius  of  curvature) 

> 1000  m 

Appendix  A,  Methods  for  Determining  Road  Dimensions  and  Geometry,  contains  the 
recommended  method  to  verify  the  geometry. 

6.2.2.2  Available  Maneuver  Room 

In  this  test,  the  warning  system  recognizes  the  road  edge  but  cannot  determine  if  there  is 
on-coming  traffic.  Therefore,  the  AMR  equals  zero. 


Available  Maneuver  Room 

Comment 

0 cm 

Not  allowed  to  cross  over  the  lane  marker 

6.2.2.3  Speed 

The  vehicle  speeds  during  the  tests  cover  the  warning  system  operating  range. 


Low  Speed 

Medium  Speed 

High  Speed 

64  km/h  (40  mph) 

89  km/h  (55  mph) 

113  km/h  (70  mph) 

6.2.2.4  Departure  Rate 

The  vehicle  departure  rates  during  the  tests  fall  within  two  ranges.  These  rates  are 
comparable  to  the  rates  suggested  in  the  ROR  report  and  in  the  ISO  standard  for  testing 
lateral  drift  warning  systems. 


Low  Departure  Rate 

High  Departure  Rate 

< 0.75  m/s 

> 0.75  m/s 

6.2.3  Measurements 

Measurements  collected  during  the  test  support  the  following: 

1 . Determine  whether  a warning  was  given 

2.  Determine  the  desired  location  of  a warning  given  the  conditions 
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3.  Determine  the  amount  of  time  the  warning  gives  driver 


6.2.3. 1 Time  of  Warning 

See  section  2.3.9. 

6.2.3.2  Longitudinal  Speed 

See  section  2.2.2. 

6.2.3.3  Distance  to  Road  Edge 

See  section  2.2.7. 

6.2.3.4  Lateral  Velocity 

See  section  2.2.4. 

6.2.4  Test  Procedure 

The  general  procedure  for  the  test  is  to  repeat  a series  of  road  departure  trajectories 

depicted  in  Figure  20. 

6.2.4. 1 Pre-Test  Procedures 

The  procedures  before  the  test  are: 

1 . Ensure  road  geometry  meets  the  test  condition.  Measure  the  geometry  if  it  is 
unknown  (see  Appendix  A,  Methods  for  Determining  Road  Dimensions  and 
Geometry). 

2.  Ensure  lane  marker  characteristics  meet  the  test  conditions  (see  sections  2.1.1- 

2.1.12). 

3.  Check  speedometer  accuracy  (see  section  2.2.2). 

4.  Verify  IMS  camera  calibration  according  to  recommendations  in  [5].  Repeat 
camera  calibration  if  camera  has  moved. 

5.  Ensure  microphone  detects  warning  system  alerts  and  driver  speech. 

6.  Ensure  recording  system  is  working  properly. 

7.  Ensure  GPS  has  sufficient  satellite  signals. 

8.  Verify  road  boundary  for  test  (i.e.,  outer  lane  marker  plus  AMR)  and  practice 
vehicle  trajectories  (speeds,  departure  rate,  etc.)  until  accomplished  safely  and 
reliably. 

6.2.4.2  Test  Procedures 

The  procedures  during  the  test  are: 

1 . Achieve  speed.  The  driver  brings  the  vehicle  to  the  prescribed  speed  and  sets  the 
cruise  control  at  the  speed. 

2.  Set  turn  signal.  The  driver  runs  one  departure  with  turn  signal  active  and  documents 
the  departure  time.  The  driver  runs  the  remaining  departures  with  the  turn  signal 
inactive. 


40 


3.  Execute  departure.  The  driver  steers  toward  the  road  boundary  at  the  prescribed 
departure  rate  (high  or  low)  until  the  road  boundary  is  crossed.  See  section  2.2.8  for 
a method  to  achieve  the  prescribed  departure  rate. 

4.  Record  departure  rate.  The  assistant  records  the  estimated  departure  rate  as  either 
low  or  high. 

5.  Go  to  start  of  next  departure  trajectory.  The  driver  steers  the  vehicle  back  onto  the 
road  for  the  next  departure 

6.  Repeat  steps  1-5  for: 

a.  Five  left  departures  in  no  passing  zone  at  the  low  departure  rate 

b.  Five  left  departures  in  no  passing  zone  at  the  high  departure  rate 

c.  Five  left  departures  in  passing  zone  at  the  low  departure  rate 

d.  Five  left  departures  in  passing  zone  at  the  high  departure  rate 

e.  One  left  departure  in  passing  zone  at  the  high  departure  rate  with  turn  signal  on. 

7.  Repeat  steps  1 - 6 for  the  middle  prescribed  speed. 

8.  Repeat  steps  1 - 6 for  the  high  prescribed  speed. 

6.2.4.3  Post-Test  Procedures 

The  procedures  after  the  test  are: 

1.  Verify  IMS  camera  orientation  has  not  changed.  Repeat  tests  if  changed. 

6.2.5  Evaluation  Methods 

Use  the  following  methods  to  evaluate  warning  system  performance. 

6.2.5.1  Rate  System  Response 

Examine  the  video  and  audio  data  and  rate  each  warning  scenario. 

For  the  no  passing  permitted  road  departure  event: 

The  rating  is  a true  positive  if  the  system  responds  with  a warning  before  the  road 
boundary  is  crossed. 

The  rating  is  a false  negative  if  the  system  does  not  respond  with  a warning  before 
the  road  boundary  is  crossed. 

For  the  passing  permitted  road  departure  event: 

The  rating  is  a true  positive  if  the  system  responds  with  a warning  before  the  road 
boundary  is  crossed  and  the  turn  signal  is  inactive. 

The  rating  is  a false  positive  if  the  system  responds  with  a warning  before  the 
road  boundary  is  crossed  and  the  turn  signal  is  active. 
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6.2.5.2  Desired  Location  of  Warning 

For  each  test  run,  the  evaluator  determines  the  desired  location  of  the  warning.  The 
desired  warning  location  varies  with  the  departure  rate  and  the  forward  speed. 

1.  Locate  the  time  of  warning  in  the  video  data  (e.g.,  the  first  video  frame  with  the  alert). 

2.  Measure  lateral  position  and  lateral  velocity  to  the  road  edge  at  the  time  of  the 
warning. 

3.  With  equation  (1)  in  section  5.1.1,  determine  the  desired  location  for  a warning  on  a 
straight  road. 

6.2.5.3  Timeliness  of  Warning 

For  each  test  run,  the  evaluator  determines  whether  the  warning  is  early,  late  or  on  time 
using  steps  described  in  section  5.1.1.  The  results  of  the  timeliness  evaluations  are 
accumulated  according  to  section  5.3. 

6.2.6  Reporting 

The  evaluator  completes  the  following  tables  with  information  from  each  test. 


Name  of  Test 

Departure  on  Un-Divided  (Two-Way) 
Straight  Road  with  No  AMR 

Date  of  Test 

Warning  System  ID 

Name  and  Version  Number 

Test  Run  Results 


Factors 

Response 

Speed 

Departure 

Direction 

Pass 

Signal 

# of 

Run 

CR 

Vf 

Vl 

ym 

yw 

T 

mph 

Rate  m/s 

Zone 

runs 

# 

T/F 

mph 

m/s 

m 

m 

OT/E/L 

40 

<0.75 

Left 

NP 

None 

5 

1 

f! 

»! 

!» 

H 

!» 

?! 

2... 

40 

>0.75 

Left 

NP 

None 

5 

40 

<0.75 

Left 

P 

None 

5 

40 

>0.75 

Left 

P 

None 

5 

40 

>0.75 

Left 

P 

ON 

1 

55 

<0.75 

Left 

NP 

None 

5 

55 

>0.75 

Left 

NP 

None 

5 

55 

<0.75 

Left 

P 

None 

5 

55 

>0.75 

Left 

P 

None 

5 

55 

>0.75 

Left 

P 

ON 

1 

70 

<0.75 

Left 

NP 

None 

5 

70 

>0.75 

Left 

NP 

None 

5 

70 

<0.75 

Left 

P 

None 

5 

70 

>0.75 

Left 

P 

None 

5 

70 

>0.75 

Left 

P 

ON 

1 

CR  = 


System  gave  correct  response  (T)  ^ __  c 

subsection  1 . 1 above  to  determine  correct  response. 
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< 


Vf  = vehicle  forward  speed 

vi  = vehicle  lateral  speed  (departure  rate) 

ym  = the  measured  lateral  distance  between  the  vehicle  front  tire  and  the  road  boundary 
at  the  time  of  warning 

yw  = the  desired  lateral  distance  at  the  time  of  warning  using  equation  (1) 

T = Timeliness  of  warning  {early,  late  or  on  time) 


Test  Run  Summary  Results  (see  section  5.3) 


Combined  Results  for 
all  runs 

#of 

runs 

# 

CR 

# 

OT 

# 

E 

# 

L 

% 

OT 

% 

E 

% 

L 

Date/time  of  test  runs 

False  Alarm  Incident  Report 


Run 

# 

Probable  Cause 

1 

Example:  shadows  on  road  misinterpreted  as  road  edge 

Lane  Change  with  Turn  Signal  Event  Evaluation 


Number  of  Lane 
Change  Tests 

Number  of  Incorrect 
Responses 

Number  of  Correct 
Responses 

# 

# 

# 

6.3  Curved  Road  (Default  minimum  AMR) 

The  Curved  Road  (default  minimum  AMR)  test  examines  the  performance  of  the  lateral 
drift  warning  function  for  road  departures  on  a curved  road.  This  scenario  is  not  part  of 
the  curve  speed  warning  function.  During  the  Curved  Road  test,  the  vehicle  travels  within 
the  established  speed  limit  for  the  roadway.  The  test  uses  three  curves  of  different  radii: 
a slow  speed  curve,  a medium  speed  curve  and  a high-speed  curve. 

In  this  scenario,  the  vehicle  travels  straight  until  a road  departure  occurs  (Figure  21).  The 
driver  uses  a guide  target  to  maintain  the  straight  road  trajectory  during  the  departure. 

The  test  area  must  have  sufficient  room  for  safe  recovery  after  the  road  departure. 

Run  the  tests  in  the  opposite  direction  to  test  a left-side  departure  (e.g.  for  a divided 
highway). 
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Figure  21.  Vehicle  trajectory  during  test 


6.3.1  Events  and  System  Response 

The  following  table  defines  the  event  for  this  test  and  the  correct  warning  signal 
response.  The  numeral  in  Figure  21  above  displays  the  proper  trajectory  for  the  event. 


Id 

Event 

Correct  Response 

1 

Crossed  solid  line 

Issue  crossed  solid  line  warning 

6.3.2  Test  Conditions 

The  conditions  for  this  test  include  the  following. 

6.3.2. 1 Road  Geometry 

The  curve  radius  for  this  test  should  be  appropriate  for  the  low  speed,  medium  speed  and 
high-speed  conditions.  The  following  equation  calculates  the  curve  radius  for  each  speed 
condition  using  lateral  acceleration  limits  (aiat)  from  section  2.3.5. 


To  simplify  setting  up  the  test,  a curve  radius  may  fall  within  a range  of  radii  by  using  the 
lateral  acceleration  limits  for  sensitivities  2 and  4. 


Road  Geometry 

Dimension 

Low  speed  curve,  64  km/h  (40  mph) 

91m-  136  m 
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Medium  speed  curve,  89  km/h  (55  mph) 

172  m - 257  m 

Low  speed  curve,  113  km/h  (70  mph) 

278  m -417  m 

Appendix  A,  Methods  for  Determining  Road  Dimensions  and  Geometry,  contains  the 
recommended  method  to  verify  the  geometry. 

6.3.2.2  Available  Maneuver  Room 

In  this  test,  the  warning  system  recognizes  the  road  edge  but  cannot  determine  if  there  is 
additional  AMR.  Therefore,  the  system  should  assume  a default  minimum  AMR. 


Available  Maneuver  Room 

Comment 

15  cm  (6  in) 

Default  minimum  AMR 

6.3.2.3  Speed 

The  vehicle  speeds  during  the  tests  cover  the  warning  system  operating  range.  The 
operator  uses  appropriate  curve  radii  as  the  trajectory  speed  changes. 


Low  Speed 

Medium  Speed 

High  Speed 

64  km/h  (40  mph) 

89  km/h  (55  mph) 

1 13  km/h  (70  mph) 

6.3.3  Measurements 

Measurements  collected  during  the  test  support  the  following: 

1 . Determine  whether  a warning  was  given 

2.  Determine  the  desired  location  of  a warning  given  the  conditions 

3 . Determine  the  amount  of  time  the  warning  gives  driver 

6.3.3. 1 Time  of  Warning 

See  section  2.3.9. 

6.3.3.2  Longitudinal  Speed 

See  section  2.2.2. 

6.3.3.3  Distance  to  Road  Edge 

See  section  2.2.7. 

6.3.3.4  Distance  to  Curve  Entry  Point 

See  section  2.2.10. 

6.3.4  Test  Procedure 

In  this  test,  the  driver  repeats  a series  of  road  departure  trajectories  as  depicted  in  Figure 
21  above. 


6.3.4. 1 Pre-Test  Procedures 

The  procedures  before  the  test  are: 
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1 . Ensure  road  geometry  meets  the  test  condition.  Measure  the  geometry  if  it  is 
unknown  (see  Appendix  A,  Methods  for  Determining  Road  Dimensions  and 
Geometry). 

2.  Ensure  lane  marker  characteristics  meet  the  test  conditions  (see  sections  2.1.1- 

2.1.12). 

3.  Check  speedometer  accuracy  (see  section  2.2.2). 

4.  Verify  IMS  camera  calibration  according  to  recommendations  in  [5].  Repeat 
camera  calibration  if  camera  has  moved. 

5.  Ensure  microphone  detects  warning  system  alerts  and  driver  speech. 

6.  Ensure  recording  system  is  working  properly. 

7.  Ensure  GPS  has  sufficient  satellite  signals. 

8.  Verify  road  boundary  for  test  (i.e.,  outer  lane  marker  plus  AMR)  and  practice 
vehicle  trajectories  (speeds,  departure  rate,  etc.)  until  accomplished  safely  and 
reliably. 

6.3.4.2  Test  Procedures 

The  procedures  during  the  test  are: 

1.  Achieve  speed.  The  driver  brings  the  vehicle  to  the  prescribed  speed  (low  speed 
curve  is  64  km/h  (40  mph)).  The  driver  uses  the  cruise  control  if  available. 

2.  Straighten  vehicle  before  curve  entry.  Before  entering  the  curve,  the  driver  centers 
the  vehicle  in  the  lane  using  the  lane  markers  as  a visual  guide.  The  vehicle  remains 
centered  into  the  curve. 

3.  Maintain  straight  trajectory  into  the  curve.  As  the  vehicle  enters  the  curve,  the 
driver  uses  a guide  target  to  maintain  a straight  trajectory  through  the  departure. 

4.  Depart  road.  The  driver  crosses  over  the  lane  marker  and  then  recovers  from  the 
departure. 

5.  Repeat  the  departure  trajectories  for: 

a.  Five  right  departures 

b.  Five  left  departures 

6.  Repeat  steps  1 - 5 for  the  middle  speed  curve. 

7.  Repeat  steps  1 - 5 for  the  high-speed  curve. 

6.3.4.3  Post-Test  Procedures 

The  procedures  after  the  test  are: 

1.  Verify  IMS  camera  orientation  has  not  changed.  Repeat  tests  if  changed. 

6.3.5  Evaluation  Methods 

Use  the  following  methods  to  evaluate  warning  system  performance. 

6.3.5.1  Rate  System  Response 

Examine  the  video  and  audio  data  and  rate  each  warning  scenario. 
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The  rating  is  a true  positive  if  the  system  responds  with  a warning  before  the  road 
boundary  is  crossed. 

The  rating  is  a false  negative  if  the  system  does  not  respond  with  a warning  before 
the  road  boundary  is  crossed. 

6.3.5.2  Desired  Location  of  Warning 

For  each  test  run,  the  evaluator  determines  the  desired  location  of  the  warning.  This  will 
vary  depending  on  the  measured  departure  rate  and  forward  speed. 

1 . Locate  the  time  of  warning  in  the  video  data  (for  example,  by  locating  video  frame 
where  alert  is  first  displayed). 

2.  Measure  lateral  position  to  the  road  edge  at  time  of  warning. 

3.  Use  equation  (4)  in  section  5. 1 .2  to  determine  the  desired  location  for  a warning  on  a 
curve. 

6.3.5.3  Timeliness  of  Warning 

For  each  test  run,  the  evaluator  determines  whether  the  warning  is  early , late  or  on  time 
using  steps  described  in  section  5.1.1.  The  results  of  the  timeliness  evaluations  are 
accumulated  according  to  section  5.3. 

6.3.6  Reporting 

The  evaluator  completes  the  following  tables  from  the  data  from  each  test. 


Name  of  Test 

Departure  on  Divided  Curved  Road  with  Fixed  Available 
Maneuver  Room 

Date  of  Test 

Warning  System  ID 

Name  and  Version  Number 

Test  Run  Results 


Facf 

ors 

Results 

Curve 

Speed 

mph 

Direction 

# of 

runs 

Run 

# 

CR 

T/F 

Vf 

mph 

ym 

m 

Xw 

m 

T 

OT/E/L 

LS 

40 

Left 

5 

1 

tl 

Vf 

vv 

2... 

LS 

40 

Right 

5 

MS 

55 

Left 

5 

MS 

55 

Right 

5 

HS 

70 

Left 

5 

HS 

70 

Right 

5 

Curve  = Low  speed  curve,  medium  speed  curve  or  high  speed  curve 
CR  = System  gave  correct  response  (T)  or  failed  to  give  correct  response  (F).  Refer  to 
subsection  1.1  above  to  determine  correct  response. 

Vf  = vehicle  forward  speed 
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ym  = the  measured  lateral  distance  between  the  vehicle  front  tire  and  the  road  boundary 
at  the  time  of  warning 

xw  = the  desired  distance  to  curve  entry  point  at  the  time  of  warning  using  equation  (4) 
T = Timeliness  of  warning  (< early , late  or  on  time ) 


Test  Run  Summary  Results  (see  section  5.3) 


Combined  Results  for 
all  runs 

# of 

runs 

# 

CR 

# 

OT 

# 

E 

# 

L 

% 

OT 

% 

E 

% 

L 

Date/time  of  test  runs 

False  Alarm  Incident  Report 


Run 

# 

Probable  Cause 

1 

Example:  shadows  on  road  misinterpreted  as  road  edge 
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7 Tests  for  Departures  on  Roads  with  Maximum  AMR 

The  Departure  on  Roads  with  Maximum  AMR  tests  evaluate  warning  system  response  to 
road  departures  where  a shoulder  provides  increased  available  maneuver  room.  These 
tests  do  not  assess  the  ability  of  the  system  to  detect  obstacles  on  the  shoulder. 

Therefore,  these  tests  do  not  provide  a complete  assessment  of  a fully  functional  warning 
system.  A warning  system  must  also  pass  the  “Tests  for  Departures  Toward  an 
Obstacle”  that  appear  in  section  8. 

These  tests  assess  the  capability  of  a warning  system  to  detect  the  shoulder  and  measure 
its  width.  The  transition  between  on-road  and  off-road  must  have  some  discriminating 
visual  property,  such  as  color  or  texture.  The  tests  employ  two  off-road  materials:  dirt 
and  artificial  grass.  The  off-road  material  lies  upon  a paved  surface  so  that  the  vehicle 
may  depart  from  the  road  with  minimal  risk. 

The  tests  employ  two  shoulder  widths.  The  large  shoulder  width  is  twice  the  maximum 
AMR.  The  system  should  detect  the  large  shoulder  but  limit  the  AMR  to  the  maximum 
in  order  to  provide  consistent  warnings.  The  small  shoulder  is  half  the  maximum  AMR. 
The  system  should  detect  the  small  shoulder  and  issue  the  warning  earlier  than  for  a large 
shoulder.  An  earlier  warning  indicates  the  system  successfully  detected  the  decreased 
shoulder  width. 

The  test  scenarios  include  departures  on  a straight  road  and  on  a curved  road. 
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7.1  Straight  Road  with  Shoulder  (Maximum  AMR) 

The  Straight  Road  with  Shoulder  test  assesses  the  warning  system’s  ability  to  detect  the 
shoulder  width  and  to  adjust  the  warning  location  on  a straight  road  departure. 


The  figure  below  illustrates  the  test  scenario.  The  left-side  departure  is  similar. 


Figure  22.  Straight  Road  with  Shoulder  Test  Trajectory 


7.1.1  Events  and  System  Response 

The  following  table  defmes  the  events  for  this  test  and  the  correct  warning  response.  See 
the  figure  above  to  locate  the  event  (e.g.,  Id  A1  is  label  1 in  figure  A). 


Id 

Event 

Correct  Response 

A1 

Road  departure 

1 . Issue  road  departure  warning 

B1 

Same  as  A1 

1.  Same  as  Al,  except  earlier 

7.1.2  Test  Conditions 

The  conditions  for  this  test  include  the  following. 

7.1.2. 1 Road  Geometry 

The  road  geometry  for  the  test  is: 


Road  Geometry 

Dimension 

Straight  (defined  using  radius  of  curvature) 

> 1000  m 
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Appendix  A,  Methods  for  Determining  Road  Dimensions  and  Geometry,  contains  the 
recommended  method  to  verify  the  road  geometry. 

7.1.2. 2  Available  Maneuver  Room 

In  this  test,  shoulder  width  affects  the  AMR.  However,  the  test  limits  the  AMR  to  a 
default  maximum  value  to  prevent  large  variations  in  warning  location  due  to  large 
variations  in  shoulder  width. 


Available  Maneuver  Room 

Comment 

1 m (39  in) 

Default  maximum  AMR 

7.1.2.3  Speed 

The  vehicle  speeds  during  the  tests  cover  the  warning  system  operating  range. 


Low  Speed 

Medium  Speed 

High  Speed 

64  km/h  (40  mph) 

89  km/h  (55  mph) 

1 13  km/h  (70  mph) 

7.1.2.4  Departure  Rate 

The  vehicle  departure  rates  during  the  tests  fall  within  two  ranges.  These  rates  are 
comparable  to  the  rates  suggested  in  the  ROR  report  and  in  the  ISO  standard  for  testing 
lateral  drift  warning  systems. 


Low  Departure  Rate 

High  Departure  Rate 

< 0.75  m/s 

> 0.75  m/s 

7.1.2.5  Shoulder  Width 

This  test  uses  two  shoulder  widths.  The  small  shoulder  is  one  half  the  default  maximum 
AMR  and  the  large  shoulder  is  twice  the  AMR.  Update  these  values  if  the  default 
minimum  or  maximum  AMR  changes. 


Shoulder  Width  Name 

Dimension 

Small  Shoulder 

0.5  m (19.5  in) 

Large  Shoulder 

2 m 

7.1.2.6  Off-Road  Material 

These  tests  use  two  types  of  material  to  simulate  off-road  surfaces.  The  operator  places 
the  material  on  a wide  shoulder  to  reduce  the  visual  width  of  the  shoulder  to  the  widths 
specified  above. 


Off-road 

Characteristics 

Dirt 

Brown  or  variations. 

Grass 

Green  artificial  grass  material 
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7.1.3  Measurements 

Measurements  collected  during  the  test  support  the  following: 

1 . Determine  whether  a warning  was  given 

2.  Determine  the  desired  location  of  a warning  given  the  conditions 

3.  Determine  the  amount  of  time  the  warning  gives  driver 

7.1.3. 1 Time  of  Warning 

See  section  2.3.9. 

7.1.3.2  Longitudinal  Speed 

See  section  2.2.2. 

7.1.3.3  Distance  to  Road  Edge 

See  section  2.2.7. 

7.1.3.4  Lateral  Velocity 

See  section  2.2.4. 

7.1.4  Test  Procedure 

The  test  repeats  a series  of  road  departure  trajectories  as  depicted  in  Figure  22. 

7.1.4. 1 Pre-Test  Procedures 

The  procedures  before  the  test  are: 

1.  Ensure  road  geometry  meets  the  test  condition.  Measure  the  geometry  if  it  is 
unknown  (see  Appendix  A,  Methods  for  Determining  Road  Dimensions  and 
Geometry). 

2.  Ensure  lane  marker  characteristics  meet  the  test  conditions  (see  sections  2.1.1- 

2.1.12). 

3.  Check  speedometer  accuracy  (see  section  2.2.2). 

4.  Verify  EMS  camera  calibration  according  to  recommendations  in  [5].  Repeat 
camera  calibration  if  camera  has  moved. 

5.  Ensure  microphone  detects  warning  system  alerts  and  driver  speech. 

6.  Ensure  recording  system  is  working  properly. 

7.  Ensure  GPS  has  sufficient  satellite  signals. 

8.  Verify  road  boundary  for  test  (i.e.,  outer  lane  marker  plus  AMR)  and  practice 
vehicle  trajectories  (speeds,  departure  rate,  etc.)  until  accomplished  safely  and 
reliably. 

9.  Set  AMR  width  with  dirt  off-road  material. 

7.1.4.2  Test  Procedures 

The  procedures  during  the  test  are: 
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1 . Achieve  speed.  The  driver  brings  the  vehicle  to  the  prescribed  speed.  The  driver 
may  use  cruise  control  if  the  vehicle  is  properly  equipped. 

2.  Execute  departure.  The  driver  steers  toward  the  road  boundary  at  the  prescribed 
departure  rate  (high  or  low)  until  the  road  boundary  (dirt  or  grass)  is  crossed.  See 
section  2.2.8  for  a method  to  achieve  the  prescribed  departure  rate. 

3.  Record  departure  rate.  The  assistant  records  the  estimated  departure  rate  as  either 
low  or  high. 

4.  Go  to  start  of  next  departure  trajectory.  The  driver  steers  the  vehicle  back  onto  the 
road  for  the  next  departure. 

5.  Repeat  steps  1-4  for: 

a.  Five  right  departures  at  the  low  departure  rate 

b.  Five  right  departures  at  the  high  departure  rate 

6.  Repeat  steps  1 - 5 for  the  prescribed  middle  speed. 

7.  Repeat  steps  1 - 5 for  the  prescribed  high  speed. 

8.  Repeat  steps  1 - 7 with  the  small  shoulder  width. 

9.  Repeat  steps  1 - 8 for  grass  off-road  material. 

7.1.4.3  Post-Test  Procedures 

The  procedures  after  the  test  are: 

1.  Verify  IMS  camera  orientation  has  not  changed.  Repeat  tests  if  changed. 

7.1.5  Evaluation  Methods 

Use  the  following  methods  to  evaluate  warning  system  performance. 

7.1.5.1  Rate  System  Response 

Examine  the  video  and  audio  data  and  rate  each  warning  scenario. 

The  rating  is  a true  positive  if  the  system  responds  with  a warning  before  the  road 
boundary  is  crossed. 

The  rating  is  a false  negative  if  the  system  does  not  respond  with  a warning  before 
the  road  boundary  is  crossed. 

7.1.5.2  Desired  Location  of  Warning 

For  each  test  run,  the  evaluator  determines  the  desired  location  of  the  warning.  The 
desired  location  varies  with  the  lane  curvature  and  the  forward  speed. 

1 . Locate  the  time  of  warning  in  the  video  data  (e.g.,  first  video  frame  with  an  alert). 

2.  Measure  lateral  position  and  lateral  velocity  to  the  road  edge  at  time  of  warning. 

3.  Use  equation  (1)  in  section  5.1.1  to  determine  the  desired  location  for  a warning  on  a 
straight  road. 
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7.1.5.3  Timeliness  of  Warning 

For  each  test  run,  the  evaluator  determines  whether  the  warning  is  early , late  or  on  time 
using  steps  described  in  section  5.1.1.  The  results  of  the  timeliness  evaluations  are 
accumulated  according  to  section  5.3. 

7.1.6  Reporting 

The  evaluator  completes  the  following  tables  from  the  data  in  each  test. 


Name  of  Test 

Departure  on  Straight  Road  with  Shoulder 

Date  of  Test 

Warning  System  ID 

Name  and  Version  Number 

Test  Run  Results 


Factors 

Results 

Offroad 

Speed 

Departure 

Direction 

Signal 

# of 

Run 

CR 

Vf 

Vl 

ym 

yw 

T 

Mat 

mph 

Rate  m/s 

runs 

# 

T/F 

mpb 

m/s 

m 

m 

OT/E/L 

Dirt 

40 

<0.75 

Left 

No 

5 

1 

ff 

ff 

H 

ff 

ff 

2... 

Dirt 

40 

<0.75 

Right 

No 

5 

Dirt 

40 

>0.75 

Left 

No 

5 

Dirt 

40 

>0.75 

Right 

No 

5 

Dirt 

40 

>0.75 

Left 

On 

1 

Dirt 

55 

<0.75 

Left 

No 

5 

Dirt 

55 

<0.75 

Right 

No 

5 

Dirt 

55 

>0.75 

Left 

No 

5 

Dirt 

55 

>0.75 

Right 

No 

5 

Dirt 

55 

>0.75 

Left 

On 

1 

Dirt 

55 

>0.75 

Right 

On 

1 

Dirt 

70 

<0.75 

Left 

No 

5 

Dirt 

70 

<0.75 

Right 

No 

5 

Dirt 

70 

>0.75 

Left 

No 

5 

Dirt 

70 

> 0.75 

Right 

No 

5 

Dirt 

70 

>0.75 

Left 

On 

1 

Dirt 

70 

>0.75 

Right 

On 

1 

Grass 

40 

<0.75 

Left 

No 

5 

Grass 

40 

<0.75 

Right 

No 

5 

Grass 

40 

>0.75 

Left 

No 

5 

Grass 

40 

>0.75 

Right 

No 

5 

Grass 

40 

>0.75 

Left 

On 

1 

Grass 

55 

<0.75 

Left 

No 

5 

Grass 

55 

<0.75 

Right 

No 

5 

Grass 

55 

>0.75 

Left 

No 

5 

Grass 

55 

>0.75 

Right 

No 

5 

Grass 

55 

>0.75 

Left 

On 

1 

Grass 

55 

>0.75 

Right 

On 

1 
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Grass 

70 

<0.75 

Left 

No 

5 

Grass 

70 

<0.75 

Right 

No 

5 

Grass 

70 

>0.75 

Left 

No 

5 

Grass 

70 

>0.75 

Right 

No 

5 

Grass 

70 

>0.75 

Left 

On 

1 

Grass 

70 

>0.75 

Right 

On 

1 

CR  = System  gave  correct  response  (T)  or  failed  to  give  correct  response  (F).  Refer  to 
subsection  1 . 1 above  to  determine  correct  response. 


Vf  = measured  vehicle  forward  speed 

vi  = measured  vehicle  lateral  speed  (departure  rate) 

ym  = measured  lateral  distance  between  the  vehicle  front  tire  and  the  road  boundary  at 
the  time  of  warning 

yw  = the  desired  lateral  distance  at  the  time  of  warning  using  equation  ( 1 ) 

T = Timeliness  of  warning  (early,  late  or  on  time) 


Test  Run  Summary  Results  (see  section  5.3) 


Combined  Results  for 
all  runs 

#of 

runs 

# 

CR 

# 

OT 

# 

E 

# 

L 

% 

OT 

% 

E 

% 

L 

Date/time  of  test  runs 

False  Alarm  Incident  Report 


Run 

# 

Probable  Cause 

1 

Example:  shadows  on  road  misinterpreted  as  road  edge 

. . . 
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7.2  Curved  Road  with  Shoulder  (Maximum  AMR) 

The  Curved  Road  with  Shoulder  test  assesses  the  warning  system’s  ability  to  detect  the 
shoulder  width  and  to  adjust  the  warning  location  on  a curved  road  departure.  This  test  is 
similar  to  the  test  in  section  7. 1 . 

The  figure  below  illustrates  the  test  scenario.  The  left-side  departure  test  uses  a right  turn 
curve. 


Figure  23.  Curved  Road  with  Shoulder  Test  Trajectory 


7.2.1  Events  and  System  Response 

The  following  table  defines  the  events  for  this  test  and  the  correct  warning  signal  the 
system  shall  provide.  See  the  figure  above  to  locate  the  event. 


Id 

Event 

Correct  Response 

A1 

Road  departure 

1 . Issue  road  departure  warning 

B1 

Same  as  A1 

1.  Same  as  A1  except  earlier 

7.2.2  Test  Conditions 

The  conditions  for  this  test  include  the  following. 
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7.2.2. 1 Road  Geometry 

The  curve  radius  for  this  test  is  appropriate  for  the  low  speed,  medium  speed  and  high- 
speed conditions.  The  following  equation  calculates  the  curve  radius  for  each  speed 

condition  using  lateral  acceleration  limits  (a/at)  from  section  2.3.5. 

? 

v 


For  the  test,  the  curve  radius  falls  within  a range  of  radii  by  using  the  lateral  acceleration 
limits  for  sensitivities  2 and  4. 


Road  Geometry 

Dimension 

Low  speed  curve,  64  km/h  (40  mph) 

91m-  136  m 

Medium  speed  curve,  89  km/h  (55  mph) 

172  m -257  m 

Low  speed  curve,  113  km/h  (70  mph) 

278  m -417  m 

Appendix  A,  Methods  for  Determining  Road  Dimensions  and  Geometry,  contains  the 
recommended  method  to  verify  the  geometry. 

7. 2.2.2  Available  Maneuver  Room 

In  this  test,  shoulder  width  affects  the  AMR.  However,  the  test  limits  the  AMR  to  a 
default  maximum  value  to  prevent  large  variations  in  warning  location  due  to  large 
variations  in  shoulder  width. 


Available  Maneuver  Room 

Comment 

1 m (39  in) 

Default  maximum  AMR 

1.223  Speed 

The  vehicle  speeds  during  the  tests  cover  the  warning  system  operating  range. 


Low  Speed 

Medium  Speed 

High  Speed 

64  km/h  (40  mph) 

89  km/h  (55  mph) 

113  km/h  (70  mph) 

7.2.2.4  Shoulder  Width 

This  test  uses  two  shoulder  widths.  The  small  shoulder  is  one  half  the  maximum  AMR 
and  the  large  shoulder  is  twice  the  AMR.  Update  these  values  if  the  default  minimum  or 
maximum  AMR  changes. 


Shoulder  Width  Name 

Shoulder  Width  Dimension 

Small  Shoulder 

0.5  m (19.5  in) 

Large  Shoulder 

2 m 

1.22.5  Off-Road  Material 

These  tests  use  two  types  of  material  to  simulate  off-road  surfaces.  The  operator  places 
the  material  on  a wide  shoulder  to  reduce  the  visual  width  of  the  shoulder  to  the  widths 
specified  above. 
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Off-road 

Characteristics 

Dirt 

Brown  or  variations. 

Grass 

Green  artificial  grass  material 

7.2.3  Measurements 

Measurements  collected  during  the  test  support  the  following: 

1 . Determine  whether  a warning  was  given 

2.  Determine  the  desired  location  of  a warning  given  the  conditions 

3.  Determine  the  amount  of  time  the  warning  gives  driver 

7.2.3. 1 Time  of  Warning 

See  section  2.3.9. 

1.23.2  Longitudinal  Speed 

See  section  2.2.2. 

7.2.3.3  Distance  to  Road  Edge 

See  section  2.2.7.  For  this  test,  the  road  edge  is  the  base  of  the  barrier. 

1.23  A Distance  to  Curve  Entry  Point 

See  section  2.2.10. 

7.2.4  Test  Procedure 

The  test  repeats  a series  of  road  departure  trajectories  as  depicted  in  Figure  23. 

7.2.4. 1 Pre-Test  Procedures 

The  procedures  before  the  test  are: 

1 . Ensure  road  geometry  meets  the  test  condition.  Measure  the  geometry  if  it  is 
unknown  (see  Appendix  A,  Methods  for  Determining  Road  Dimensions  and 
Geometry). 

2.  Ensure  lane  marker  characteristics  meet  the  test  conditions  (see  sections  2.1.1- 

2.1.12). 

3.  Check  speedometer  accuracy  (see  section  2.2.2). 

4.  Verify  IMS  camera  calibration  according  to  recommendations  in  [5].  Repeat 
camera  calibration  if  camera  has  moved. 

5.  Ensure  microphone  detects  warning  system  alerts  and  driver  speech. 

6.  Ensure  recording  system  is  working  properly. 

7.  Ensure  GPS  has  sufficient  satellite  signals. 

8.  Verify  road  boundary  for  test  (i.e.,  outer  lane  marker  plus  AMR)  and  practice 
vehicle  trajectories  (speeds,  departure  rate,  etc.)  until  accomplished  safely  and 
reliably. 

9.  Set  the  AMR  width  with  dirt  off-road  material. 
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7.2.4.2  Test  Procedures 

The  procedures  during  the  test  are: 

1 . Achieve  speed.  The  driver  brings  the  vehicle  to  the  prescribed  speed.  The  driver 
may  use  cruise  control  if  the  vehicle  is  properly  equipped. 

2.  Straighten  vehicle  and  center  vehicle  in  the  lane  before  entering  the  curve.  The 
driver  centers  the  vehicle  in  the  lane  using  the  lane  markers  as  a visual  guide.  The 
vehicle  remains  in  the  center  until  the  start  of  the  curve. 

3.  Maintain  straight  trajectory  into  the  curve.  As  the  vehicle  enters  the  curve,  the 
driver  uses  a guide  target  to  maintain  the  straight  lane  trajectory  through  the 
departure. 

4.  Depart  road.  The  driver  crosses  over  the  lane  marker  and  then  recovers  from  the 
departure. 

5.  Repeat  steps  1-4  for: 

a.  Five  right  departures. 

b.  Five  left  departures. 

6.  Repeat  steps  1 - 5 for  the  middle  speed  curve. 

7.  Repeat  steps  1 - 5 for  the  high-speed  curve. 

8.  Repeat  steps  1 - 7 with  the  small  shoulder  width. 

9.  Repeat  steps  1 - 8 with  the  grass  off-road  material. 

7.2.4.3  Post-Test  Procedures 

The  procedures  after  the  test  are: 

1 . Verify  IMS  camera  orientation  has  not  changed.  Repeat  tests  if  changed. 

7.2.5  Evaluation  Methods 

Use  the  following  methods  to  evaluate  warning  system  performance. 

7.2.5.1  Rate  System  Response 

Examine  the  video  and  audio  data  and  rate  each  warning  scenario. 

The  rating  is  a true  positive  if  the  system  responds  with  a warning  before  the  road 
boundary  is  crossed. 

The  rating  is  a false  negative  if  the  system  does  not  respond  with  a warning  before 
the  road  boundary  is  crossed. 

7.2. 5.2  Desired  Location  of  Warning 

For  each  test  run,  the  evaluator  determines  the  desired  location  of  the  warning.  The 
desired  location  varies  with  the  departure  rate  and  forward  speed. 

1 . Locate  the  time  of  warning  in  the  video  data  (for  example,  by  locating  video  frame 
where  alert  is  first  displayed). 

2.  Measure  lateral  position  and  lateral  velocity  to  the  road  edge  at  time  of  warning. 
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3.  Use  equation  (4)  in  section  5.1.2  to  determine  the  desired  location  for  a warning  on  a 
curve. 

7.2.5.3  Timeliness  of  Warning 

For  each  test  run,  the  evaluator  determines  whether  the  warning  is  early , late  or  on  time 
using  steps  described  in  section  5.1.1.  The  results  of  the  timeliness  evaluations  are 
accumulated  according  to  section  5.3. 

7.2.6  Reporting 

The  evaluator  completes  the  following  tables  from  the  data  in  each  test. 


Name  of  Test 

Departure  on  Curved  Road  with  Shoulder 

Date  of  Test 

Warning  System  ID 

Name  and  Version  Number 

Test  Run  Results 


Factors 

Results 

Offroad 

Curve 

Speed 

Direction 

# of 

Run 

CR 

Vf 

ym 

xw 

T 

Mat 

mph 

runs 

# 

T/F 

mph 

m 

m 

OT/E/L 

Dirt 

LS 

40 

Left 

5 

1 

vv 

?! 

If 

If 

2... 

Dirt 

LS 

40 

Right 

5 

Dirt 

MS 

55 

Left 

5 

Dirt 

MS 

55 

Right 

5 

Dirt 

HS 

70 

Left 

5 

Dirt 

HS 

70 

Right 

5 

Grass 

LS 

40 

Left 

5 

Grass 

LS 

40 

Right 

5 

Grass 

MS 

55 

Left 

5 

Grass 

MS 

55 

Right 

5 

Grass 

HS 

70 

Left 

5 

Grass 

HS 

70 

Right 

5 

Curve 
CR  = 


Vf  = 

ym  = 

Xw  = 
T = 


= Low  speed  curve,  medium  speed  curve  and  high  speed  curve 

System  gave  correct  response  (T)  or  failed  to  give  correct  response  (F).  Refer  to 

subsection  1.1  above  to  determine  correct  response. 

vehicle  forward  speed 

the  measured  lateral  distance  between  the  vehicle  front  tire  and  the  road  boundary 
at  the  time  of  warning 

the  desired  distance  to  curve  entry  point  at  the  time  of  warning  using  equation  (4) 
Timeliness  of  warning  {early,  late  or  on  time) 


Test  Run  Summary  Results  (see  section  5.3) 


Combined  Results  for 
all  runs 

# of 

runs 

# 

CR 

# 

OT 

# 

E 

# 

L 

% 

OT 

% 

E 

% 

L 

Date/time  of  test  runs 
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False  Alarm  Incident  Report 


Run 

# 

Probable  Cause 

1 

Example:  shadows  on  road  misinterpreted  as  road  edge 
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8 Tests  for  Departures  Toward  an  Obstacle 

These  tests  evaluate  warning  system  response  to  road  departures  toward  an  obstacle  on 
the  shoulder.  The  warning  system  detects  the  obstacle,  decreases  the  available  maneuver 
room,  and  issues  a warning  based  on  the  location  of  the  obstacles.  These  tests  do  not 
assess  the  system’s  ability  to  determine  whether  the  shoulder  ends  before  the  obstacle 
(i.e..,  the  obstacle  is  off-road).  Therefore,  a warning  system  must  also  pass  the  “Tests  for 
Departures  on  Roads  with  Maximum  AMR”  that  appear  in  section  7. 

These  tests  evaluate  the  system's  ability  to  detect  an  obstacle  and  to  warn  when  there  is  a 
potential  for  a lateral  collision.  A map  may  provide  information  regarding  obstacles,  but 
this  information  may  be  incomplete,  inaccurate  or  unavailable.  Thus,  a warning  system 
must  use  a real-time  sensor  (e.g.,  radar)  to  detect  obstacles.  The  warning  system  deals 
with  longitudinal  barriers  and  discrete  objects. 

A longitudinal  barrier  appears  as  one  long  continuous  obstacle,  such  as  a jersey  barrier. 
The  vehicle  drifts  slowly  toward  the  barrier  much  like  a lane  departure.  The  warning 
system  uses  a side-looking  sensor  to  detect  these  obstacles. 

A discrete  object  appears  on  the  shoulder  ahead  of  the  vehicle.  Examples  of  discrete 
obstacles  include  parked  cars,  bridge  supports,  and  the  beginning  of  a longitudinal 
barrier.  If  the  vehicle  drifts  too  far  outside  the  lane,  it  will  collide  with  the  object.  After 
a warning,  the  driver  steers  away  from  the  obstacle  to  avoid  a collision.  The  system  uses 
a side-forward  sensor  (i.e.,  aimed  forward  and  to  the  side)  to  detect  these  obstacles. 

The  sensors  used  by  the  system  must  detect  a wide  range  of  obstacles  with  varying 
properties.  Small  obstacles,  such  as  pedestrians  and  motorcycles,  pose  the  greater 
challenges.  The  tests  use  similar  sized  obstacles  made  of  safer  material  to  reduce  the  risk 
of  serious  damage. 

The  tests  involve  placing  the  obstacles  at  two  distances  (i.e.,  offsets)  from  the  lane  edge. 
The  large  offset  is  twice  the  maximum  AMR.  The  system  should  detect  the  large 
obstacle  offset  but  limit  the  AMR  to  the  maximum  in  order  to  provide  consistent 
warnings.  The  small  obstacle  offset  is  half  the  maximum  AMR.  The  system  should 
detect  the  small  offset  and  adjust  the  warning  accordingly.  The  earlier  warning  indicates 
the  system  successfully  detected  the  obstacle. 

The  test  scenarios  include  obstacles  on  a straight  road  and  on  a curved  road. 
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8.1  Barrier  on  Straight  Road 

The  Barrier  on  Straight  Road  test  assesses  the  warning  system’s  ability  to  detect  the 
location  of  a barrier  and  to  adjust  the  warning  location  on  a straight  road  departure. 


The  figure  below  illustrates  the  test  scenario.  Left-side  collision  tests  position  the  barrier 
outside  of  the  yellow  line. 


Figure  24.  Barrier  on  Straight  Road  Test,  Vehicle  Trajectory 


8.1.1  Events  and  System  Response 

The  following  table  defmes  the  events  for  this  test  and  the  correct  system  response.  See 
the  figure  above  to  locate  the  event  (e.g.,  Id  A1  is  label  1 in  figure  A). 


Id 

Event 

Correct  Response 

A1 

Lateral  collision 

1 . Issue  lateral  collision  warning 

B1 

Lateral  collision 

1.  Same  as  A 1 

8.1.2  Test  Conditions 

The  conditions  for  this  test  include  the  following. 

8. 1.2.1  Road  Geometry 

The  road  geometry  for  the  test  is: 
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Road  Geometry 

Dimension 

Straight  (defined  using  radius  of  curvature) 

> 1000  m 

Appendix  A,  Methods  for  Determining  Road  Dimensions  and  Geometry,  contains  the 
recommended  method  to  verify  the  road  geometry. 

8.1.2.2  Available  Maneuver  Room 

In  this  test,  obstacle  offset  affects  the  AMR.  However,  the  test  limits  the  AMR  to  a 
default  maximum  value  to  prevent  large  variations  in  warning  location  due  to  large 
variations  in  obstacle  offset. 


Available  Maneuver  Room 

Comment 

1 m (39  in) 

Maximum  AMR 

8.1.2.3  Speed 

The  vehicle  speeds  during  the  tests  cover  the  warning  system  operating  range. 


Low  Speed 

Medium  Speed 

High  Speed 

64  km/h  (40  mph) 

89  km/h  (55  mph) 

113  km/h  (70  mph) 

8.1.2.4  Departure  rate 

The  vehicle  departure  rates  during  the  tests  fall  within  two  ranges.  These  rates  are 
comparable  to  the  rates  suggested  in  the  ROR  report  and  in  the  ISO  standard  for  testing 
lateral  drift  warning  systems. 


Low  Departure  Rate 

High  Departure  Rate 

< 0.75  m/s 

> 0.75  m/s 

8.1.2.5  Barrier 

For  safety  purposes,  this  test  uses  a plastic,  water-filled  jersey  barrier  as  shown  in  Figure 
25. 
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Figure  25.  Water-filled  Plastic  Jersey  Barrier 


8.1.2.6  Barrier  Offset 

The  tests  involve  placing  the  obstacles  at  two  offsets  measured  from  the  outside  edge  of 
the  lane  marker.  The  small  offset  is  one  half  the  maximum  AMR  and  the  large  offset  is 
twice  the  AMR.  The  system  evaluator  updates  these  values  if  the  default  minimum  or 
maximum  AMR  changes. 


Barrier  Offset  Name 

Dimension 

Small  Offset 

0.5  m (19.5  in) 

Large  Offset 

2 m (79  in) 

8.1.3  Measurements 

Measurements  collected  during  the  test  support  the  following: 

1 . Determine  whether  a warning  was  given 

2.  Determine  the  desired  location  of  a warning  given  the  conditions 

3.  Determine  the  amount  of  time  the  warning  gives  driver 

8.1.3. 1 Time  of  Warning 

See  section  2.3.9. 

8.1.3.2  Longitudinal  Speed 

See  section  2.2.2. 

8. 1.3.3  Distance  to  Road  Edge 

See  section  2.2.7.  For  this  test,  the  road  edge  is  the  base  of  the  barrier. 

8.1. 3.4  Lateral  Velocity 

See  section  2.2.4. 
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8.1.4  Test  Procedure 

The  general  procedure  for  the  test  repeats  a series  of  road  departure  trajectories  depicted 
in  Figure  24. 

8.1.4.1  Pre-Test  Procedures 

The  procedures  before  the  test  are: 

1 . Ensure  road  geometry  meets  the  test  condition  and  measure  the  geometry  if 
unknown  (see  Appendix  A,  Methods  for  Determining  Road  Dimensions  and 
Geometry). 

2.  Ensure  lane  marker  characteristics  meet  the  test  conditions  and  measure  the 
characteristics  if  it  is  unknown  (see  sections  2.1.1-2.1.12). 

3.  Check  speedometer  accuracy  (see  section  2.2.2). 

4.  Verify  IMS  camera  calibration  according  to  recommendations  in  [5]  and  repeat 
camera  calibration  if  camera  has  moved. 

5.  Ensure  microphone  detects  warning  system  alerts  and  driver  speech. 

6.  Ensure  recording  system  is  working  properly. 

7.  Ensure  GPS  has  sufficient  satellite  signals. 

8.  Assemble  the  barriers  at  the  desired  offset  (large  distance  for  first  test  set). 

9.  Verify  road  boundary  for  test  (i.e.,  outer  lane  marker  plus  AMR)  and  practice 
vehicle  trajectories  (speeds,  departure  rate,  etc.)  until  accomplished  safely  and 
reliably. 

8.1.4.2  Test  Procedures 

The  procedures  during  the  test  are: 

1 . Achieve  speed.  The  driver  brings  the  vehicle  to  the  prescribed  speed.  The  driver 
may  use  cruise  control  if  the  vehicle  is  properly  equipped. 

2.  Execute  departure.  The  driver  steers  toward  the  barrier  at  the  prescribed  departure 
rate  (high  or  low)  until  a warning  occurs  or  the  driver  decides  to  avoid  the  barrier. 
Section  2.2.8  provides  a method  to  achieve  the  prescribed  departure  rate. 

3.  Record  departure  rate.  The  assistant  records  the  estimated  departure  rate  as  either 
low  or  high. 

4.  Go  to  start  of  next  collision  trajectory.  The  driver  brings  the  vehicle  back  around  in 
preparation  for  the  next  collision  trajectory. 

5.  Repeat  steps  1-4  for: 

a.  Five  right-side  collisions  at  the  low  departure  rate. 

b.  Five  left-side  collisions  at  the  low  departure  rate. 

c.  Five  right-side  collisions  at  the  high  departure  rate. 

d.  Five  left-side  collisions  at  the  high  departure  rate. 

6.  Repeat  steps  1 - 5 for  the  prescribed  middle  speed. 

7.  Repeat  steps  1 - 5 for  the  prescribed  high  speed. 

8.  Repeat  steps  1-7  with  the  barriers  at  the  small  offset. 

8.1.4.3  Post-Test  Procedures 


66 


The  procedures  after  the  test  are: 

1.  Verify  IMS  camera  orientation  has  not  changed.  Repeat  tests  if  changed. 

8.1.5  Evaluation  Methods 

Use  the  following  methods  to  evaluate  warning  system  performance. 

8.1.5.1  Rate  System  Response 

Examine  the  video  and  audio  data  and  rate  each  warning  scenario. 

The  rating  is  a true  positive  if  the  system  responds  with  a warning  before  the  road 
boundary  is  crossed. 

The  rating  is  a false  negative  if  the  system  does  not  respond  with  a warning  before 
the  road  boundary  is  crossed. 

8.1.5.2  Desired  Location  of  Warning 

For  each  test  run,  the  evaluator  determines  the  desired  location  of  the  warning.  The 
desired  location  varies  with  the  forward  speed  and  curve  radii. 

1 . Locate  the  time  of  warning  in  the  video  data  (for  example,  by  locating  video  frame 
where  alert  is  first  displayed). 

2.  Measure  lateral  position  and  lateral  velocity  to  the  road  edge  at  time  of  warning. 

3.  Use  equation  (1)  in  section  5.1.1  to  determine  the  desired  location  for  a warning  on  a 
straight  road. 

8.1.5.3  Timeliness  of  Warning 

For  each  test  run,  the  evaluator  determines  whether  the  warning  is  early , late  or  on  time 
using  steps  described  in  section  5.1.1.  The  results  of  the  timeliness  evaluations  are 
accumulated  according  to  section  5.3. 

8.1.6  Reporting 

The  evaluator  completes  the  following  tables  from  the  data  in  each  test. 


Name  of  Test 

Collision  with  Barrier  on  Straight  Road 

Date  of  Test 

Warning  System  ID 

Name  and  Version  Number 

Test  Run  Results 


Factors 

Results 

Offset 

m 

Speed 

mph 

Departure 
Rate  m/s 

Direction 

Signal 

# of 

runs 

Run 

# 

CR 

T/F 

Vf 

mph 

Vl 

m/s 

ym 

m 

yw 

m 

T 

OT/E/L 

0.5 

40 

<0.75 

Left 

No 

5 

1 

0.5 

ff 

ff 

ff 

ff 

2... 

0.5 

40 

<0.75 

Right 

No 

5 
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0.5 

40 

>0.75 

Left 

No 

5 

0.5 

40 

>0.75 

Right 

No 

5 

0.5 

40 

>0.75 

Left 

On 

1 

0.5 

55 

<0.75 

Left 

No 

5 

0.5 

55 

<0.75 

Right 

No 

5 

0.5 

55 

>0.75 

Left 

No 

5 

0.5 

55 

>0.75 

Right 

No 

5 

0.5 

55 

>0.75 

Left 

On 

1 

0.5 

55 

>0.75 

Right 

On 

1 

0.5 

70 

<0.75 

Left 

No 

5 

0.5 

70 

<0.75 

Right 

No 

5 

0.5 

70 

>0.75 

Left 

No 

5 

0.5 

70 

>0.75 

Right 

No 

5 

0.5 

70 

>0.75 

Left 

On 

1 

0.5 

70 

>0.75 

Right 

On 

1 

2.0 

40 

<0.75 

Left 

No 

5 

2.0 

40 

<0.75 

Right 

No 

5 

2.0 

40 

>0.75 

Left 

No 

5 

2.0 

40 

>0.75 

Right 

No 

5 

2.0 

40 

>0.75 

Left 

On 

1 

2.0 

55 

<0.75 

Left 

No 

5 

2.0 

55 

<0.75 

Right 

No 

5 

2.0 

55 

>0.75 

Left 

No 

5 

2.0 

55 

>0.75 

Right 

No 

5 

2.0 

55 

>0.75 

Left 

On 

1 

2.0 

55 

>0.75 

Right 

On 

1 

2.0 

70 

<0.75 

Left 

No 

5 

2.0 

70 

<0.75 

Right 

No 

5 

2.0 

70 

>0.75 

Left 

No 

5 

2.0 

70 

>0.75 

Right 

No 

5 

2.0 

70 

>0.75 

Left 

On 

1 

2.0 

70 

>0.75 

Right 

On 

1 

Offset  = Offset  distance  (in  meters)  of  barrier  from  outside  edge  of  lane  marker 
CR=  System  gave  correct  response  (T)  or  failed  to  give  correct  response  (F).  Refer  to 
subsection  1.1  above  to  determine  correct  response. 

Vf  = measured  vehicle  forward  speed 

vi  = measured  vehicle  lateral  speed  (departure  rate) 

ym  = measured  lateral  distance  between  the  vehicle  front  tire  and  the  road  boundary  at 
the  time  of  warning 

yw  = the  desired  lateral  distance  at  the  time  of  warning  using  equation  (1) 

T = Timeliness  of  warning  {early,  late  or  on  time) 


Test  Run  Summary  Result 

s (see  section  5.3 

) 

Combined  Results  for 

# of 

# 

# 

# 

# 

% 

% 

% 

all  runs 

runs 

CR 

OT 

E 

L 

OT 

E 

L 
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Date/time  of  test  runs 


False  Alarm  Incident  Report 


Run 

# 

Probable  Cause 

1 

Example:  shadows  on  road  misinterpreted  as  road  edge 
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8.2  Barrier  on  Curved  Road 

The  Barrier  on  Curved  Road  test  assesses  the  warning  system’s  ability  to  detect  the 
location  of  a barrier  and  to  adjust  the  warning  location  on  a curved  road  departure. 

The  figure  below  illustrates  the  test  scenario.  The  left-side  collision  avoidance  test  uses  a 
right  turn  curve. 


Figure  26.  Barrier  on  Curved  Road  Test  Vehicle  Trajectory 


8.2.1  Events  and  System  Response 

The  following  table  defines  the  events  for  this  test  and  the  correct  system  response.  See 
the  figure  above  to  locate  the  event  (e.g.,  Id  A1  is  label  1 in  figure  A). 


Id 

Event 

Correct  Response 

A1 

Lateral  collision 

1 . Issue  lateral  collision  warning 

B1 

Lateral  collision 

1.  Same  as  A1  except  earlier 

8.2.2  Test  Conditions 

The  conditions  for  this  test  include  the  following. 
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8.2.2. 1  Road  Geometry 

The  curv  e radius  for  this  test  should  be  appropriate  for  the  low  speed,  medium  speed  and 
high-speed  conditions.  The  following  equation  calculates  the  curve  radius  for  each  speed 
condition  using  lateral  acceleration  limits  (aial)  from  section  2.3.5. 

2 

V 


For  the  test,  the  curve  radius  falls  within  a range  of  radii  by  using  the  lateral  acceleration 
limits  for  sensitivities  2 and  4. 


Road  Geometry 

Dimension 

Low  speed  curve,  64  km/h  (40  mph) 

91m-  136  m 

Medium  speed  curve,  89  km/h  (55  mph) 

172  m - 257  m 

Low  speed  curve,  1 13  km/h  (70  mph) 

278  m -417  m 

Appendix  A,  Methods  for  Determining  Road  Dimensions  and  Geometry,  contains  the 
recommended  method  to  verify  the  road  geometry. 

8.2.2.2  Available  Maneuver  Room 

In  this  test,  obstacle  offset  affects  the  AMR.  However,  the  test  limits  the  AMR  to  a 
default  maximum  value  to  prevent  large  variations  in  warning  location  due  to  large 
variations  in  obstacle  offset. 


Available  Maneuver  Room 

Comment 

1 m (39  in) 

Maximum  AMR 

8.2.2.3  Speed 

The  vehicle  speeds  during  the  tests  cover  the  warning  system  operating  range. 


Low  Speed 

Medium  Speed 

High  Speed 

64  km/h  (40  mph) 

89  km/h  (55  mph) 

1 13  km/h  (70  mph) 

8.2.2.4  Barrier 

For  safety  purposes,  this  test  uses  a plastic,  water-filled  jersey  barrier  (see  Figure  25). 

8.2.2.5  Barrier  Offset 

The  tests  involve  placing  the  obstacles  at  two  offsets  measured  from  the  outside  edge  of 
the  lane  marker.  The  small  offset  is  one  half  the  maximum  AMR  and  the  large  offset  is 
twice  the  AMR.  The  test  operator  updates  these  values  if  the  default  minimum  or 
maximum  AMR  changes. 


Barrier  Offset  Name 

Dimension 

Small  Offset 

0.5  m (19.5  in) 

Large  Offset 

2 m (79  in) 
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8.2.3  Measurements 

Measurements  collected  during  the  test  support  the  following: 

1 . Determine  whether  a warning  was  given 

2.  Determine  the  desired  location  of  a warning  given  the  conditions 

3.  Determine  the  amount  of  time  the  warning  gives  driver 

8.2.3. 1 Time  of  Warning 

See  section  2.3.9. 

8.2.3.2  Longitudinal  Speed 

See  section  2.2.2. 

8.2.3.3  Distance  to  Road  Edge 

See  section  2.2.7.  For  this  test,  the  road  edge  is  the  base  of  the  barrier. 

8.2.3.4  Distance  to  Curve  Entry  Point 

See  section  2.2.10. 

8.2.4  Test  Procedure 

The  general  procedure  for  the  test  repeats  a series  of  road  departure  trajectories  depicted 
in  Figure  26. 

8.2.4.1  Pre-Test  Procedures 

The  procedures  before  the  test  are: 

1 . Ensure  road  geometry  meets  the  test  condition.  The  operator  measures  the  road 
geometry  as  per  Appendix  A if  required. 

2.  Ensure  the  lane  marker  characteristics  meet  the  test  conditions.  The  operator 
measures  the  characteristics  if  it  is  unknown  (see  sections  2.1.1-2.1.12). 

3.  Check  speedometer  accuracy  (see  section  2.2.2). 

4.  Verify  IMS  camera  calibration  according  to  recommendations  in  [5].  Repeat 
camera  calibration  if  camera  has  moved. 

5.  Ensure  microphone  detects  warning  system  alerts  and  driver  speech. 

6.  Ensure  recording  system  is  working  properly. 

7.  Ensure  the  GPS  has  sufficient  satellites  and  operates  properly. 

8.  Assemble  the  barriers  and  position  for  test  (use  large  offset  for  first  test  set). 

9.  Verify  road  boundary  for  test  (i.e.,  outer  lane  marker  plus  AMR)  and  practice 
vehicle  trajectories  (speeds,  departure  rate,  etc.)  until  accomplished  safely  and 
reliably. 

8.2.4.2  Test  Procedures 

The  procedures  during  the  test  are: 
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1.  Achieve  speed.  The  driver  brings  the  vehicle  to  the  prescribed  speed  (low  speed 
curve  is  64  km/h  (40  mph)).  The  driver  may  use  cruise  control  if  the  vehicle  is 
properly  equipped. 

2.  Straighten  and  center  vehicle  before  entering  curve.  The  driver  centers  the  vehicle 
with  the  lane  markers  as  a visual  guide.  The  vehicle  remains  in  the  center  until  the 
start  of  the  curve. 

3.  Maintain  straight  trajectory  into  the  curve.  As  the  vehicle  enters  the  curve,  the 
driver  uses  a guide  target  to  maintain  a straight  trajectory  toward  the  collision. 

4.  Collide  with  obstacle.  The  driver  continues  straight  until  a warning  takes  place  or 
the  driver  aborts  before  a collision. 

5.  Repeat  steps  1-4  for: 

a.  Five  right-side  collisions. 

b.  Five  left-side  collisions. 

6.  Repeat  steps  1 - 5 for  the  collision  on  the  middle  speed  curve. 

7.  Repeat  steps  1 - 5 for  the  collision  on  the  high-speed  curve. 

8.  Repeat  steps  1 - 7 with  the  obstacles  at  the  small  offset. 

8.2.4.3  Post-Test  Procedures 

The  procedures  after  the  test  are: 

1.  Verify  IMS  camera  orientation  has  not  changed.  Repeat  tests  if  changed. 

8.2.5  Evaluation  Methods 

Use  the  following  methods  to  evaluate  warning  system  performance. 

8.2.5. 1 Rate  System  Response 

Examine  the  video  and  audio  data  and  rate  each  warning  scenario. 

The  rating  is  a true  positive  if  the  system  responds  with  a warning  before  the  road 
boundary  is  crossed. 

The  rating  is  a false  negative  if  the  system  does  not  respond  with  a warning  before 
the  road  boundary  is  crossed. 

8.2. 5.2  Desired  Location  of  Warning 

For  each  test  run,  the  evaluator  determines  the  desired  location  of  the  warning.  The 
desired  location  varies  with  the  forward  speed  and  curve  radii. 

1 . Locate  the  time  of  warning  in  the  video  data  (for  example,  by  locating  video  frame 
where  alert  is  first  displayed). 

2.  Measure  lateral  position  and  lateral  velocity  to  the  road  edge  at  time  of  warning. 

3.  Use  equation  (4)  in  section  5. 1 .2  to  determine  the  desired  location  for  a warning  on  a 
curve. 
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8.2.5.3  Timeliness  of  Warning 

The  evaluator  determines  the  timeliness  of  each  warning  (early,  late  or  on  time)  and 
compiles  statistics  according  to  section  5.3. 

8.2.6  Reporting 

The  evaluator  completes  the  following  tables  from  the  data  in  each  test. 


Name  of  Test 

Collision  with  Barrier  on  Curved  Road 

Date  of  Test 

Warning  System  ID 

Name  and  Version  Number 

Test  Run  Results 


Factors 

Results 

Offset 

Curve 

Speed 

Direction 

#of 

Run 

CR 

Vf 

ym 

T 

m 

mph 

runs 

# 

T/F 

mph 

m 

m 

OT/E/L 

0.075 

LS 

40 

Left 

5 

1 

If 

If 

If 

If 

2... 

0.075 

LS 

40 

Right 

5 

0.075 

MS 

55 

Left 

5 

0.075 

MS 

55 

Right 

5 

0.075 

HS 

70 

Left 

5 

0.075 

HS 

70 

Right 

5 

2.0 

LS 

40 

Left 

5 

2.0 

LS 

40 

Right 

5 

2.0 

MS 

55 

Left 

5 

2.0 

MS 

55 

Right 

5 

2.0 

HS 

70 

Left 

5 

2.0 

HS 

70 

Right 

5 

Offset  = Offset  distance  (in  meters)  of  barrier  from  outside  edge  of  lane  marker 
Curve  = Low  speed  curve,  medium  speed  curve  and  high  speed  curve 
CR=  System  gave  correct  response  (T)  or  failed  to  give  correct  response  (F).  Refer  to 
subsection  1 . 1 above  to  determine  correct  response. 

Vf  = vehicle  forward  speed 

ym  = the  measured  lateral  distance  between  the  vehicle  front  tire  and  the  road  boundary 
at  the  time  of  warning 

xw  - the  desired  distance  to  curve  entry  point  at  the  time  of  warning  using  equation  (4) 
T = Timeliness  of  warning  (early,  late  or  on  time) 


Test  Run  Summary  Results  (see  section  5.3) 


Combined  Results  for 
all  runs 

#of 

runs 

# 

CR 

# 

OT 

# 

E 

# 

L 

% 

OT 

% 

E 

% 

L 

Date/time  of  test  runs 

False  Alarm  Incident  Report 


Run  Probable  Cause 
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# 

1 

Example:  shadows  on  road  misinterpreted  as  road  edge 

• • . 
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8.3  Obstacle  on  Shoulder  on  Straight  Road 

This  test  assesses  the  warning  system’s  ability  to  detect  an  obstacle  on  the  shoulder  ahead 
of  the  vehicle  and  to  adjust  the  warning  location  on  a straight  road  departure. 

The  figure  below  illustrates  the  test  scenario.  Reverse  the  vehicle’s  direction  to  test  a 
left-side  collision. 


Figure  27.  Obstacle  on  Shoulder  on  Straight  Road  Test,  Vehicle  Trajectory 

8.3.1  Events  and  System  Response 

The  following  table  defines  the  events  for  this  test  and  the  correct  system  response.  See 
the  figure  above  to  locate  the  event  (e.g.,  Id  A1  is  label  1 in  figure  A). 


Id 

Event 

Correct  Response 

A1 

Lateral  collision 

1 . Issue  lateral  collision  warning 

B1 

Lateral  collision 

1.  Same  as  A1  except  earlier 

8.3.2  Test  Conditions 

The  conditions  for  this  test  include  the  following. 

8.3.2. 1 Road  Geometry 

The  road  geometry  for  the  test  is: 


Road  Geometry 

Dimension 

Straight  (defined  using  radius  of  curvature) 

> 1000  m 
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Appendix  A,  Methods  for  Determining  Road  Dimensions  and  Geometry,  contains  the 
recommended  method  to  verify  the  road  geometry. 

8.3.2.2  Available  Maneuver  Room 

In  this  test,  obstacle  offset  determines  the  AMR.  However,  the  AMR  is  limited  to  a 
maximum  value  to  prevent  large  variations  warning  location  caused  by  large  variations 
shoulder  width. 


Available  Maneuver  Room 

Comment 

1 m (39  in) 

Maximum  AMR 

8.3.2.3  Speed 

The  vehicle  speeds  during  the  tests  cover  the  warning  system  operating  range. 


Low  Speed 

Medium  Speed 

High  Speed 

64  km/h  (40  mph) 

89  km/h  (55  mph) 

113  km/h  (70  mph) 

8.3.2.4  Departure  rate 

The  vehicle  departure  rates  during  the  tests  fall  within  two  ranges.  These  rates  are 
comparable  to  the  rates  suggested  in  the  ROR  report  and  in  the  ISO  standard  for  testing 
lateral  drift  warning  systems. 


Low  Departure  Rate 

High  Departure  Rate 

< 0.75  m/s 

> 0.75  m/s 

8.3.2.5  Obstacle 

For  safety  purposes,  this  test  uses  a plastic,  traffic  barrel  (see  Figure  28). 


Figure  28.  Traffic  barrel 


77 


8.3.2.6  Obstacle  Offset 

The  tests  involve  placing  the  obstacles  at  two  offsets  measured  from  the  outside  edge  of 
the  lane  marker.  The  small  offset  is  one  half  the  maximum  AMR  and  the  large  offset  is 
twice  the  AMR.  The  operator  updates  these  values  if  the  default  minimum  or  maximum 
AMR  changes.. 


Barrier  Offset  Name 

Dimension 

Small  Offset 

0.5  m (19.5  in) 

Large  Offset 

2 m (79  in) 

8.3.3  Measurements 

Measurements  collected  during  the  test  support  the  following: 

1 . Determine  whether  a warning  was  given 

2.  Determine  the  desired  location  of  a warning  given  the  conditions 

3.  Determine  the  amount  of  time  the  warning  gives  driver 

8.3.3. 1 Time  of  Warning 

See  section  2.3.9. 

8.3.3.2  Longitudinal  Speed 

See  section  2.2.2. 

5.3.3.3  Distance  to  Road  Edge 

See  section  2.2.7.  For  this  test,  the  base  of  the  obstacle  defines  the  road  edge.  The 
obstacle  will  not  be  adjacent  to  the  front  wheel  at  the  time  of  warning.  The  evaluator 
calculates  the  distance  to  the  road  edge  using  the  distance  to  the  lane  marker  plus  the 
obstacle  offset  (see  figure  below).  To  take  into  account  the  departure  angle  ( <9  requires 
departure  rate,  see  section  2.2.9),  the  evaluator  uses  the  following  equation  (see  Figure 
29): 

yw=l  + h=l  + 

COS# 

Where: 

yw  = the  lateral  distance  between  the  vehicle  front  tire  and  the  road  boundary 
/ = the  measured  lateral  distance  to  the  lane  marker 

o = the  known  offset  of  the  obstacle 
6 = the  departure  angle 
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Vehicle  location 


Figure  29.  Indirect  Distance  to  Road  Edge 


8.33.4  Lateral  Velocity 

See  section  2.2.4. 

8.3.4  Test  Procedure 

The  general  procedure  for  the  test  repeats  a series  of  road  departure  trajectories  depicted 

in  Figure  27. 

8.3.4. 1 Pre-Test  Procedures 

The  procedures  before  the  test  consist  of  the  following  steps: 

1 . Ensure  road  geometry  meets  the  test  condition  and  measure  the  geometry  if 
unknown  (see  Appendix  A,  Methods  for  Determining  Road  Dimensions  and 
Geometry). 

2.  Ensure  lane  marker  characteristics  meet  the  test  conditions  and  measure  the 
characteristics  if  unknown  (see  sections  2.1.1-2.1.12). 

3.  Check  speedometer  accuracy  (see  section  2.2.2). 

4.  Verify  IMS  camera  calibration  according  to  recommendations  in  [5].  Repeat 
camera  calibration  if  camera  has  moved. 

5.  Ensure  microphone  detects  warning  system  alerts  and  driver  speech. 

6.  Ensure  recording  system  is  working  properly. 

7.  Ensure  GPS  has  sufficient  satellite  signals. 

8.  Assemble  the  barrels  at  the  desired  offset  (large  distance  for  first  test  set). 

9.  Verify  road  boundary  for  test  (i.e.,  outer  lane  marker  plus  AMR)  and  practice 
vehicle  trajectories  (speeds,  departure  rate,  etc.)  until  accomplished  safely  and 
reliably. 

83.4.2  Test  Procedures 

The  procedures  during  the  test  are: 

1 . Achieve  speed.  The  driver  brings  the  vehicle  to  the  prescribed  speed.  The  driver 
may  use  cruise  control  if  the  vehicle  is  properly  equipped. 

2.  Execute  departure.  The  driver  steers  toward  the  obstacle  at  the  prescribed  departure 
rate  (high  or  low)  until  a warning  occurs  or  the  driver  decides  to  avoid  the  obstacle. 
Section  2.2.8  provides  a method  to  achieve  the  prescribed  departure  rate. 
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3.  Go  to  start  of  next  collision  trajectory.  The  driver  brings  the  vehicle  back  around  in 
preparation  for  the  next  collision  trajectory. 

4.  Repeat  steps  1-3  for: 

a.  Five  right-side  collisions  at  the  low  departure  rate. 

b.  Five  left-side  collisions  at  the  low  departure  rate. 

c.  Five  right-side  collisions  at  the  high  departure  rate. 

d.  Five  left-side  collisions  at  the  high  departure  rate. 

5.  Repeat  steps  1-4  for  the  prescribed  middle  speed. 

6.  Repeat  steps  1 - 4 for  the  prescribed  high  speed. 

7.  Repeat  steps  1-6  with  the  barriers  at  the  large  offset. 

8.3.4.3  Post-Test  Procedures 

The  procedures  after  the  test  are: 

1.  Verify  IMS  camera  orientation  has  not  changed.  Repeat  tests  if  changed. 

8.3.5  Evaluation  Methods 

Use  the  following  methods  to  evaluate  warning  system  performance. 

8.3.5.1  Rate  System  Response 

Examine  the  video  and  audio  data  and  rate  each  warning  scenario. 

The  rating  is  a true  positive  if  the  system  responds  with  a warning  before  the  road 
boundary  is  crossed. 

The  rating  is  a false  negative  if  the  system  does  not  respond  with  a warning  before 
the  road  boundary  is  crossed. 

8.3.5.2  Desired  Location  of  Warning 

For  each  test  run,  the  evaluator  determines  the  desired  location  of  the  warning.  This  will 
vary  depending  on  the  measured  departure  rate  and  forward  speed. 

1 . Locate  the  time  of  warning  in  the  video  data  (for  example,  by  locating  video  frame 
where  alert  is  first  displayed). 

2.  Measure  lateral  position  and  lateral  velocity  to  the  road  edge  at  time  of  warning. 

3.  Use  equation  (1)  in  section  5.1.1  to  determine  the  desired  location  for  a warning  on  a 
straight  road. 

8.3.5.3  Timeliness  of  Warning 

For  each  test  run,  the  evaluator  determines  whether  the  warning  is  early , late  or  on  time 
using  steps  described  in  section  5.1.1.  The  results  of  the  timeliness  evaluations  are 
accumulated  according  to  section  5.3. 

8.3.6  Reporting 

The  evaluator  completes  the  following  tables  from  the  data  in  each  test. 
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Name  of  Test 

Collision  with  Obstacle  on  Shoulder  on  Straight  Road 

Date  of  Test 

Warning  System  ID 

Name  and  Version  Number 

Test  Run  Results 


Factors 

Results 

Offset 

Speed 

Departure 

Direction 

Signal 

#of 

Run 

CR 

Vf 

Vl 

ym 

yw 

T 

m 

mph 

Rate  m/s 

runs 

# 

T/F 

mph 

m/s 

m 

m 

OT/E/L 

0.5 

40 

<0.75 

Left 

No 

5 

1 

0.5 

ft 

ft 

ft 

ff 

2... 

0.5 

40 

<0.75 

Right 

No 

5 

0.5 

40 

>0.75 

Left 

No 

5 

0.5 

40 

>0.75 

Right 

No 

5 

0.5 

40 

>0.75 

Left 

On 

1 

0.5 

55 

<0.75 

Left 

No 

5 

0.5 

55 

<0.75 

Right 

No 

5 

0.5 

55 

>0.75 

Left 

No 

5 

0.5 

55 

>0.75 

Right 

No 

5 

0.5 

55 

>0.75 

Left 

On 

1 

0.5 

55 

>0.75 

Right 

On 

1 

0.5 

70 

<0.75 

Left 

No 

5 

0.5 

70 

<0.75 

Right 

No 

5 

0.5 

70 

>0.75 

Left 

No 

5 

0.5 

70 

>0.75 

Right 

No 

5 

0.5 

70 

>0.75 

Left 

On 

1 

0.5 

70 

>0.75 

Right 

On 

1 

2.0 

40 

<0.75 

Left 

No 

5 

2.0 

40 

<0.75 

Right 

No 

5 

2.0 

40 

>0.75 

Left 

No 

5 

2.0 

40 

>0.75 

Right 

No 

5 

2.0 

40 

>0.75 

Left 

On 

1 

2.0 

55 

<0.75 

Left 

No 

5 

2.0 

55 

<0.75 

Right 

No 

5 

2.0 

55 

>0.75 

Left 

No 

5 

2.0 

55 

>0.75 

Right 

No 

5 

2.0 

55 

>0.75 

Left 

On 

1 

2.0 

55 

>0.75 

Right 

On 

1 

2.0 

70 

<0.75 

Left 

No 

5 

2.0 

70 

<0.75 

Right 

No 

5 

2.0 

70 

>0.75 

Left 

No 

5 

2.0 

70 

>0.75 

Right 

No 

5 

2.0 

70 

>0.75 

Left 

On 

1 

2.0 

70 

>0.75 

Right 

On 

1 

Offset 

= 01 

'fset  distance  (in  meters)  of  obstacle 

'rom  outsic 

e edge  of  lane  marker 
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CR  = System  gave  correct  response  (T)  or  failed  to  give  correct  response  (F).  Refer  to 
subsection  1 . 1 above  to  determine  correct  response. 

Vf  = measured  vehicle  forward  speed 

vi  = measured  vehicle  lateral  speed  (departure  rate) 

ym  = measured  lateral  distance  between  the  vehicle  front  tire  and  the  road  boundary  at 
the  time  of  warning 

yw  = the  desired  lateral  distance  at  the  time  of  warning  using  equation  (1) 

T = Timeliness  of  warning  {early,  late  or  on  time) 


Test  Run  Summary  Results  (see  section  5.3) 


Combined  Results  for 
all  runs 

#of 

runs 

# 

CR 

# 

OT 

# 

E 

# 

L 

% 

OT 

% 

E 

% 

L 

Date/time  of  test  runs 

False  Alarm  Incident  Report 


Run 

# 

Probable  Cause 

1 

Example:  shadows  on  road  misinterpreted  as  road  edge 

. . . 

• . • 
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8.4  Obstacle  on  Shoulder  on  Curved  Road 

The  Obstacle  on  Shoulder  on  Curved  Road  test  assesses  the  warning  system’s  ability  to 
detect  an  obstacle  on  the  shoulder  ahead  of  the  vehicle  and  to  adjust  the  warning  location 
on  a curved  road  departure. 


The  figure  below  illustrates  the  test  scenario.  The  left-side  collision  avoidance  test  uses  a 
right  turn  curve. 


o 


A B 

Figure  30.  Obstacle  on  Shoulder  on  Curved  Road  Test,  Vehicle  trajectory 

8.4.1  Events  and  System  Response 

The  following  table  defines  the  events  for  this  test  and  the  correct  system  response.  See 
the  figure  above  to  locate  the  event  (e.g.,  Id  A1  is  label  1 in  figure  A). 


Id 

Event 

Correct  Response 

A1 

Lateral  collision 

1 . Issue  lateral  collision  warning 

B1 

Lateral  collision 

1.  Same  as  A 1 

8.4.2  Test  Conditions 

The  conditions  for  this  test  include  the  following. 
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8.4.2. 1  Road  Geometry 

The  curve  radius  for  this  test  should  be  appropriate  for  the  low  speed,  medium  speed  and 
high-speed  conditions.  The  following  equation  calculates  the  curve  radius  for  each  speed 
condition  using  lateral  acceleration  limits  (aiat)  from  section  2.3.5. 


For  the  test,  the  curve  radius  falls  within  a range  of  radii  by  using  the  lateral  acceleration 
limits  for  sensitivities  2 and  4. 


Road  Geometry 

Dimension 

Low  speed  curve,  64  km/h  (40  mph) 

91m-  136  m 

Medium  speed  curve,  89  km/h  (55  mph) 

172  m -257  m 

Low  speed  curve,  1 13  km/h  (70  mph) 

278  m - 417  m 

Appendix  A,  Methods  for  Determining  Road  Dimensions  and  Geometry,  contains  the 
recommended  method  to  verify  the  road  geometry. 

8.4.2.2  Maximum  Available  Maneuver  Room 

ha  this  test,  obstacle  offset  determines  the  AMR.  However,  the  AMR  is  limited  to  a 
maximum  value  to  prevent  large  variations  warning  location  caused  by  large  variations 
shoulder  width. 


Available  Maneuver  Room 

Comment 

1 m (39  in) 

Maximum  AMR 

8.4.2.3  Speed 

The  vehicle  speeds  during  the  tests  cover  the  warning  system  operating  range. 


Low  Speed 

Medium  Speed 

High  Speed 

64  km/h  (40  mph) 

89  km/h  (55  mph) 

113  km/h  (70  mph) 

8.4.2.4  Obstacle 

For  safety  purposes,  this  test  uses  a plastic,  traffic  barrel  (see  Figure  28). 

8.4.2.5  Obstacle  Offset 

The  tests  involve  placing  the  obstacles  at  two  offsets  measured  from  the  outside  edge  of 
the  lane  marker.  The  small  offset  is  one  half  the  maximum  AMR  and  the  large  offset  is 
twice  the  AMR.  The  test  operator  updates  these  values  if  the  default  minimum  or 
maximum  AMR  changes. 


Barrier  Offset  Name 

Dimension 

Small  Offset 

0.5  m (19.5  in) 

Large  Offset 

2 m (79  in) 
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8.4.3  Measurements 

Measurements  determine  whether  a warning  was  given,  the  desired  location  of  a warning 
given  the  conditions  and  the  amount  of  time  the  warning  gives  driver. 

8.4.3. 1 Time  of  Warning 

See  section  2.3.9. 

8.4.3.2  Longitudinal  Speed 

See  section  2.2.2. 

8.4.3.3  Distance  to  Road  Edge 

See  section  2.2.7.  For  this  test,  the  base  of  the  obstacle  defines  the  road  edge.  If  the 
obstacle  is  not  adjacent  to  the  front  wheel  at  the  time  of  warning  and  a direct 
measurement  is  not  possible,  then  the  evaluator  calculates  the  distance  to  the  road  edge 
using  the  distance  to  the  lane  marker  plus  the  obstacle  offset  (see  Figure  29).  To  take  into 
account  the  departure  angle  (^requires  departure  rate,  see  section  2.2.9),  use  the 
following  equation: 


Where: 

yw  = the  lateral  distance  between  the  vehicle  front  tire  and  the  road  boundary 
/ = the  measured  lateral  distance  to  the  lane  marker 

o = the  known  offset  of  the  obstacle 
6 = the  departure  angle 

8.4.3.4  Distance  to  Curve  Entry  Point 

See  section  2.2.10. 

8.4.4  Test  Procedure 

The  general  procedure  for  the  test  repeats  a series  of  road  departure  trajectories  depicted 
in  Figure  30. 

8.4.4. 1 Pre-Test  Procedures 

The  procedures  before  the  test  are: 

1 . Ensure  road  geometry  meets  the  test  condition.  The  operator  measures  the  road 
geometry  as  per  Appendix  A if  required. 

2.  Ensure  the  lane  marker  characteristics  meet  the  test  conditions.  The  operator 
measures  the  characteristics  if  it  is  unknown  (see  sections  2.1.1-2.1.12). 

3.  Check  speedometer  accuracy  (see  section  2.2.2). 

4.  Verify  IMS  camera  calibration  according  to  recommendations  in  [5].  Repeat 
camera  calibration  if  camera  has  moved. 

5.  Ensure  microphone  detects  warning  system  alerts  and  driver  speech. 

6.  Check  Ensure  recording  system  is  working  properly. 
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7.  Ensure  the  GPS  has  sufficient  satellites  and  operates  properly. 

8.  Assemble  the  barrels  and  position  for  tests  (use  large  offset  for  first  test). 

9.  Verify  road  boundary  for  test  (i.e.,  outer  lane  marker  plus  AMR)  and  practice 
vehicle  trajectories  (speeds,  departure  rate,  etc.)  until  accomplished  safely  and 
reliably. 

8.4.4.2  Test  Procedures 

The  procedures  during  the  test  are: 

1.  Achieve  speed.  The  driver  brings  the  vehicle  to  the  prescribed  speed  (low  speed 
curve  is  64  km/h  (40  mph))  and  sets  the  cruise  control  at  the  speed. 

2.  Straighten  and  center  vehicle  before  entering  curve.  The  driver  centers  the  vehicle 
with  the  lane  markers  as  a visual  guide.  The  vehicle  remains  in  the  center  until  the 
start  of  the  curve. 

3.  Maintain  straight  trajectory  into  the  curve.  As  the  vehicle  enters  the  curve,  the 
driver  uses  the  obstacle  as  a guide  to  maintain  a straight  trajectory  toward  the 
collision. 

4.  Collide  with  obstacle.  The  driver  continues  straight  until  a warning  takes  place  or 
the  driver  aborts  before  collision. 

5.  Repeat  steps  1-4  for: 

a.  Five  right-side  collisions. 

b.  Five  left-side  collisions. 

6.  Repeat  steps  1 - 5 for  the  collision  on  the  middle  speed  curve. 

7.  Repeat  steps  1 - 5 for  the  collision  on  the  high-speed  curve. 

8.  Repeat  steps  1 - 7 with  the  obstacles  at  the  large  offset. 

8.4.4.3  Post-Test  Procedures 

The  procedures  after  the  test  are: 

1.  Verify  IMS  camera  orientation  has  not  changed.  Repeat  tests  if  changed. 

8.4.5  Evaluation  Methods 

Use  the  following  methods  to  evaluate  warning  system  performance. 

8.4.5. 1 Rate  System  Response 

Examine  the  video  and  audio  data  and  rate  each  warning  scenario. 

The  rating  is  a true  positive  if  the  system  responds  with  a warning  before  the  road 
boundary  is  crossed. 

The  rating  is  a false  negative  if  the  system  does  not  respond  with  a warning  before  the 
road  boundary  is  crossed. 
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8.4. 5.2  Desired  Location  of  Warning 

For  each  test  run,  the  evaluator  determines  the  desired  location  of  the  warning.  The 
desired  location  varies  with  the  forward  speed  and  curve  radii. 

1 . Locate  the  time  of  warning  in  the  video  data  (for  example,  by  locating  video  frame 
where  alert  is  first  displayed). 

2.  Measure  lateral  position  and  lateral  velocity  to  the  road  edge  at  time  of  warning. 

3.  Use  equation  (4)  in  section  5. 1 .2  to  determine  the  desired  location  for  a warning  on  a 
curve. 

8.4. 5.3  Timeliness  of  Warning 

The  evaluator  determines  the  timeliness  of  each  warning  (early,  late  or  on  time)  and 
compiles  statistics  according  to  section  5.3. 

8.4.6  Reporting 

The  evaluator  completes  the  following  tables  from  the  data  in  each  test. 


Name  of  Test 

Collision  with  Obstacle  on  Curved 

Road 

Date  of  Test 

Warning  System  ID 

Name  and  Version  Number 

Test  Run  Results 


Factors 

Results 

Offset 

Curve 

Speed 

Direction 

#of 

Run 

CR 

Vf 

Vm 

Xw 

T 

m 

mph 

runs 

# 

T/F 

mph 

m 

m 

OT/E/L 

0.075 

LS 

40 

Left 

5 

1 

ft 

ft 

ft 

ft 

2... 

0.075 

LS 

40 

Right 

5 

0.075 

MS 

55 

Left 

5 

0.075 

MS 

55 

Right 

5 

0.075 

HS 

70 

Left 

5 

0.075 

HS 

70 

Right 

5 

2.0 

LS 

40 

Left 

5 

2.0 

LS 

40 

Right 

5 

2.0 

MS 

55 

Left 

5 

2.0 

MS 

55 

Right 

5 

2.0 

HS 

70 

Left 

5 

2.0 

HS 

70 

Right 

5 

Offset  = Offset  distance  (in  meters)  of  obstacle  from  outside  edge  of  lane  marker 
Curve  = Low  speed  curve,  medium  speed  curve  and  high  speed  curve 
CR  = System  gave  correct  response  (T)  or  failed  to  give  correct  response  (F).  Refer  to 
subsection  1 . 1 above  to  determine  correct  response. 

Vf  = vehicle  forward  speed 
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ym  = the  measured  lateral  distance  between  the  vehicle  front  tire  and  the  road  boundary 
at  the  time  of  warning 

xw  = the  desired  distance  to  curve  entry  point  at  the  time  of  warning  using  equation  (4) 
T = Timeliness  of  warning  {early,  late  or  on  time) 


Test  Run  Summary  Results  (see  section  5.3) 


Combined  Results  for 
all  runs 

#of 

runs 

# 

CR 

# 

OT 

# 

E 

# 

L 

% 

OT 

% 

E 

% 

L 

Date/time  of  test  runs 

False  Alarm  Incident  Report 


Run 

# 

Probable  Cause 

1 

Example:  shadows  on  road  misinterpreted  as  road  edge 

. . . 

. . . 
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9 Tests  for  Departures  on  Curves  at  Excessive  Speed 

The  tests  for  Departures  on  Curves  at  Excessive  Speed  evaluate  the  warning  system’s 
ability  to  detect  whether  the  vehicle  is  traveling  too  fast  for  an  approaching  curve  and  to 
warn  the  driver  in  time  to  slow  the  vehicle  to  a safe  speed.  Since  the  curve  may  be 
beyond  visual  limits,  curve-speed  warnings  rely  on  a map  to  obtain  the  curvature  of 
upcoming  road  segments  and  a positioning  system  (e.g.,  Global  Positioning  System)  to 
provide  the  vehicle  location  relative  to  the  curve.  The  system  estimates  the  amount  of 
deceleration  required  to  slow  the  vehicle  to  a safe  speed  given  the  distance  to  the  curve. 
The  safe  speed  takes  into  account  the  radius  of  the  curve  and  other  available  data  such  as 
temperature  and  precipitation  (e.g.,  windshield  wiper  on  or  off),  which  provides  an 
indirect  measure  of  road  friction.  The  speeds  and  curve  radii  are  consistent  with  a rural 
road  scenario  where  this  type  of  crash  most  often  occurs. 

The  curve-branch  test  examines  the  system’s  ability  to  correctly  determine  whether  the 
vehicle  is  taking  the  curve  or  straight  branch  when  a road  network  splits.  Curve-speed 
warnings  for  the  curve-branch  of  the  road  are  a nuisance  when  the  vehicle  is  heading 
toward  the  straight  branch  of  the  road. 
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9.1  Excessive  Speed  on  Curved  Road 

The  Excessive  Speed  on  Curved  Road  test  examines  the  performance  of  the  curve  speed 
warning  system  for  the  scenario  where  a vehicle  is  traveling  too  fast  for  an  upcoming 
curve  in  the  road.  This  scenario  falls  under  the  "negotiating  a curve  on  undivided  non- 
freeway with  two  lanes  at  non-junction"  scenario  identified  in  the  NHTSA  test  scenario 
report  as  one  of  the  major  settings  for  road  departure  crashes  [3].  This  test  assumes  the 
warning  system  uses  GPS  and  a digital  map  to  determine  the  distance  to  approaching 
curves  and  the  radius  of  the  curve. 

The  figure  below  illustrates  the  test  scenario. 
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Figure  31.  Excessive  Speed  on  Curved  Road  Test,  Vehicle  Trajectory 


9.1.1  Events  and  System  Response 

The  following  table  defines  the  events  for  this  test  and  the  correct  warning  signal  the 
system  shall  provide.  The  primary  event  is  the  vehicle  reaching  a distance  from  a curve 
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where  the  current  speed  is  too  fast  for  the  driver  to  safely  negotiate  the  curve.  See  the 
figure  above  to  locate  the  event. 


Id 

Event 

Correct  Response 

1 

Excessive  speed  for  curve 

1 . Issue  excessive  speed  for  curve  warning 

9.1.2  Test  Conditions 

The  conditions  for  this  test  include  the  following. 

9.1.2. 1 Road  Geometry 

The  test  examines  two  types  of  curves  suggested  in  [3].  A range  of  curve  radii  provides 
flexibility  in  selecting  suitable  test  sites. 


Road  Type 

Dimension 

Non-freeway 

50  m - 100  m 

Freeway 

200  m - 350  m 

Appendix  A,  Methods  for  Determining  Road  Dimensions  and  Geometry,  contains  the 
recommended  method  to  verify  the  road  geometry. 

9.1.2.2  Temperature 

The  temperature  condition  for  the  test  falls  within  the  following  ranges: 


Condition 

Temperature 

Cold 

0°  C - 3°  C (32°  F - 38°  F) 

Warm 

4°  C - 7°  C (39°  F - 45°  F) 

Artificial  means  to  achieve  the  temperature  at  the  sensor  are  acceptable  (e.g.,  ice). 

9.1. 2.3  Precipitation 

The  precipitation  during  the  test  is  either  dry  or  wet.  The  warning  system  determines 
precipitation  from  windshield  wiper  status. 


Condition 

Windshield  Wiper 

Dry 

Off 

Wet 

On 

9. 1.2.4  Speed 

The  test  requires  the  driver  to  approach  the  curve  at  a speed  in  excess  of  the  safe  speed 
for  the  curve.  Use  equation  (6)  to  compute  the  safe  speed  for  the  curve.  Use  3 m/s2  (0.3 
g)  lateral  acceleration  limit  (sensitivity  = 3)  for  warm,  dry  condition.  Use  1 m/s2  (0.1  g) 
for  cold,  wet  condition  ([12]  page  10.27).  The  following  table  provides  an  example  of 
the  speeds  to  use  for  a test  on  two  radii  chosen  for  freeway  and  non-freeway  roads. 
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Road  Type 

Curve 

Radius 

Condition 

Safe  speed 

Over- 
speed 1 
(115  %) 

Over- 
speed 2 
(130  %) 

Freeway 

200  m 

Warm,  dry 

87  km/h 
(54  mph) 

100  km/h 
(62  mph) 

1 13  km/h 
(70  mph) 

Cold,  wet 

50  km/h 
(31  mph) 

58  km/h 
(36  mph) 

68  km/h 
(41  mph) 

Non- 

Freeway 

75  m 

Warm,  dry 

53  km/h 
(33  mph) 

61  km/h 
(38  mph) 

33  km/h 
(38  mph) 

Cold,  wet 

31  km/h 
(19  mph) 

35  km/h 
(22  mph) 

40  km/h 
(25  mph) 

9.1.3  Measurements 

Measurements  collected  during  the  test  support  the  following: 

1 . Determine  whether  a warning  was  given 

2.  Determine  the  desired  location  of  a warning  given  the  conditions 

3.  Determine  the  amount  of  time  the  warning  gives  driver 

9.1.3. 1 Time  of  Warning 

See  section  2.3.9. 

9.1.3.2  Longitudinal  Speed 

See  section  2.2.2. 

9.1.3.3  Distance  to  Curve  Entry  Point 

See  section  2.2.10. 

9.1.4  Test  Procedure 

The  general  procedure  for  the  test  repeats  a series  of  road  departure  trajectories  depicted 
in  Figure  31. 

9.1.4. 1 Pre-Test  Procedures 

The  procedures  before  the  test  are: 

1 . Ensure  road  geometry  meets  the  test  condition  and  measure  the  geometry  as  per 
Appendix  A if  required. 

2.  Ensure  the  lane  marker  characteristics  meet  the  test  conditions.  The  operator 
measures  the  characteristics  if  it  is  unknown  (see  sections  2.1.1-2.1.12). 

3.  Check  speedometer  accuracy  (see  section  2.2.2). 

4.  Verify  IMS  camera  calibration  according  to  recommendations  in  [5].  Repeat 
camera  calibration  if  camera  has  moved. 

5.  Ensure  microphone  detects  warning  system  alerts  and  driver  speech. 
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6.  Ensure  recording  system  is  working  properly. 

7.  Ensure  the  GPS  has  sufficient  satellites  and  is  operating  properly. 

8.  Establish  proper  initial  sensor  conditions  (initial  test  is  a warm,  dry,  non-freeway 
scenario). 

9.  Verify  curve  entry  point  for  test  (i.e.,  outer  lane  marker  plus  AMR)  and  practice 
vehicle  trajectories  (speeds,  decelerations,  etc.)  until  accomplished  safely  and 
reliably.  Ensure  recovery  area  exists  in  case  driver  is  not  able  to  keep  vehicle  in 
lane  through  curve. 

9. 1.4.2  Test  Procedures 

The  procedures  during  the  test  are: 

1.  Achieve  temperature.  Set  the  system’s  temperature  sensor  too  output  prescribed 
condition  (warm  or  cold). 

2.  Achieve  precipitation  condition.  Set  the  windshield  wiper  to  the  prescribed 
condition  (dry  = off,  wet  = on). 

3.  Achieve  speed.  The  driver  brings  the  vehicle  to  the  over-speed  condition. 

4.  Maintain  vehicle  centered  in  lane  until  past  curve  entry  point.  Vehicle  may  depart 
out  of  lane  into  recovery  area  if  traveling  too  fast. 

5.  Document  the  warning  type  and  result. 

6.  Repeat  steps  1-  5 five  times. 

7.  Repeat  steps  1 - 6 for  over- speed  2 (130  % of  maximum) 

8.  Repeat  steps  1 - 7 for  cold,  wet  condition 

9.  Repeat  steps  1 - 8 for  freeway 

9.1.4.3  Post-Test  Procedures 

The  procedures  after  the  test  are: 

1 . Verify  IMS  camera  orientation  has  not  changed.  Repeat  tests  if  changed. 

9.1.5  Evaluation  Methods 

Use  the  following  methods  to  evaluate  warning  system  performance. 

9. 1.5.1  Rate  System  Response 

Examine  the  video  and  audio  data  and  rate  each  warning  scenario. 

The  rating  is  a true  positive  if  the  system  responds  with  a warning  before  the  vehicle  has 
completed  travel  through  the  curve. 

The  rating  is  a false  negative  if  the  system  does  not  respond  with  a warning  before  the 
vehicle  exits  the  curve. 

9. 1.5. 2 Desired  Location  of  Warning 

For  each  test  run,  the  evaluator  determines  the  desired  location  of  the  warning. 
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1 . Locate  the  time  of  warning  in  the  video  data  (for  example,  by  locating  video  frame 
where  alert  is  first  displayed). 

2.  Measure  distance  to  curve  entry  at  time  of  warning. 

3.  Use  equation  (7)  in  section  5.2  to  determine  the  desired  location  for  a warning  when 
traveling  too  fast  for  an  approaching  curve. 

9.1.5.3  Timeliness  of  Warning 

For  each  test  run,  the  evaluator  determines  whether  the  warning  is  early , late  or  on  time 
using  steps  described  in  section  5.2.  The  results  of  the  timeliness  evaluations  are 
accumulated  according  to  section  5.3. 

9.1.6  Reporting 

The  evaluator  completes  the  following  tables  from  the  data  in  each  test. 


Name  of  Test 

Curve  Speed  Warning  Test 

Date  of  Test 

Warning  System  ID 

Name  and  Version  Number 

Test  Run  Results 


Factors 

Results 

Condition 

Speed 

#of 

Run 

CR 

Vf 

Xm 

Xw 

T 

mph 

runs 

# 

T/F 

mph 

m 

m 

OT/E/ 

L 

Warm/Dry 

67 

10 

1 

(i 

?! 

2... 

Cold/Wet 

52 

10 

1 

CR=  System  gave  correct  response  (T)  or  failed  to  give  correct  response  (F).  Refer  to 
subsection  1 . 1 above  to  determine  correct  response. 

Vf  = measured  vehicle  forward  speed 

xm  = The  measured  distance  to  the  curve  at  the  time  of  warning 

xw  = the  desired  distance  to  the  curve  at  the  time  of  warning  using  equation  (7) 

T = Timeliness  of  warning  {early,  late  or  on  time ) 


Test  Run  Summary  Results  (see  section  5.3) 


Combined  Results  for 
all  runs 

# of 

runs 

# 

CR 

# 

OT 

# 

E 

# 

L 

% 

OT 

% 

E 

% 

L 

Date/time  of  test  runs 

False  Alarm  Incident  Report 


Run 

# 

Probable  Cause 

1 

Example:  shadows  on  road  misinterpreted  as  road  edge 

2 

Example:  longitudinal  road  joint  misinterpreted  as  road  edge 
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9.2  Curve-Branch  Detection 

The  Curve-Branch  Detection  test  examines  whether  the  curve-speed  warning  system 
accurately  predicts  whether  a vehicle  is  about  to  travel  on  a straight  or  curved  branch  of  a 
road  that  splits.  This  test  assumes  the  warning  system  uses  GPS  and  a digital  map  to 
determine  the  distance  to  approaching  curves  and  the  radius  of  the  curve.  The  scenario 
for  this  test  requires  a two- lane  road  that  splits  into  a straight  road  and  a curved  road. 

The  curve-branch  requires  sufficient  radius  to  generate  a warning  at  a reasonable 
approach  speed.  For  test  purposes,  the  rural  road  curve  used  in  the  curve-speed  warning 
test  may  be  adapted  for  the  curve-branch  test. 

During  the  curve-branch  test,  the  system  issues  a warning  if  the  vehicle  is  in  the  lane 
headed  toward  the  curve  at  a high  speed  (see  B in  figure  below).  Conversely,  the  system 
does  not  issue  a warning  if  the  vehicle  is  in  the  lane  headed  straight  (see  A in  figure 
below).  The  test  verifies  correct  operation  in  both  lanes.  The  driver  uses  the  procedure 
in  section  9.1  for  the  test  in  the  curve  lane  and  the  procedure  below  for  the  straight  lane. 


Room  for 


Room  for 


A 


B 


Figure  32.  Curve-Branch  Test,  Vehicle  trajectory 
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9.2.1  Events  and  System  Response 

The  following  table  defines  the  events  for  this  test  and  the  correct  warning  signal  the 
system  shall  provide.  The  primary  event  is  the  vehicle  reaching  a distance  from  a curve 
where  the  current  speed  is  too  fast  for  the  driver  to  safely  negotiate  the  curve.  See  the 
figure  above  to  locate  the  event. 


Id 

Event 

Correct  Response 

A1 

Reached  Curve 

1 . No  warning  should  be  given 

9.2.2  Test  Conditions 

The  conditions  for  this  test  include  the  following. 

9.2.2. 1  Road  Geometry 

The  road  geometry  for  this  test  consists  of  a straight  lane  adjacent  to  a curved  lane.  See 
section  9. 1.2.1  for  the  curved  lane  radii.  It  is  sufficient  to  test  the  freeway  curve  only. 
The  straight  lane  has  the  following  geometry: 


Road  Geometry 

Dimension 

Straight  lane  (radius) 

> 1000 m 

Appendix  A,  Methods  for  Determining  Road  Dimensions  and  Geometry,  contains  the 
recommended  method  to  verify  the  road  geometry. 

9.2.2.2  Temperature 

Use  same  temperature  conditions  as  in  section  9. 1.2.2. 

9.2.2.3  Precipitation 

Use  same  precipitation  conditions  as  in  section  9. 1.2.3. 

9.2.2.4  Speed 

The  speeds  in  the  straight  lane  are  the  same  as  those  for  the  curved  lane.  See  section 
9. 1.2.4. 

9.2.3  Measurements 

Measurements  determine  whether  a warning  was  given  and  whether  the  test  speed  was 
within  limits. 

9.2.3. 1 Time  of  Warning 

See  section  2.3.9. 

9.2.3.2  Longitudinal  Speed 

See  section  2.2.2. 
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9.2.4  Test  Procedure 

The  general  procedure  for  the  test  repeats  a series  of  road  departure  trajectories  depicted 
in  Figure  32.  The  driver  conducts  this  test  after  the  curve  over-speed  test. 

9.2.4. 1 Pre-Test  Procedures 

The  procedures  before  the  test  are: 

1 . Ensure  road  geometry  meets  the  test  condition  and  measure  the  geometry  as  per 
Appendix  A if  required. 

2.  Ensure  the  lane  marker  characteristics  meet  the  test  conditions.  The  operator 
measures  the  characteristics  if  it  is  unknown  (see  sections  2.1.1-2.1.12). 

3.  Check  speedometer  accuracy  (see  section  2.2.2). 

4.  Verify  IMS  camera  calibration  according  to  recommendations  in  [5].  Repeat 
camera  calibration  if  camera  has  moved. 

5.  Ensure  microphone  detects  warning  system  alerts  and  driver  speech. 

6.  Ensure  recording  system  is  working  properly. 

7.  Ensure  the  GPS  has  sufficient  satellites  and  is  operating  properly. 

8.  Establish  proper  initial  sensor  conditions  (initial  test  is  a warm,  dry,  non-freeway 
scenario) 

9.2.4.2  Test  Procedures 

The  procedures  during  the  test  are: 

1 . Achieve  temperature.  Set  the  system’s  temperature  sensor  to  prescribed  condition. 

2.  Achieve  precipitation  condition.  Set  the  windshield  wiper  to  the  prescribed 
condition  (dry  = off,  wet  = on). 

3.  Achieve  speed.  The  driver  brings  the  vehicle  to  the  over-speed  condition. 

4.  Maintain  vehicle  centered  in  lane  until  past  the  split. 

5.  Document  the  warning  type  and  result, 
a.  Repeat  steps  1-5  five  times. 

6.  Repeat  steps  1 - 5 for  cold,  wet  condition 

9.2.4.3  Post-Test  Procedures 

The  procedures  after  the  test  are: 

1.  Verify  IMS  camera  orientation  has  not  changed.  Repeat  tests  if  changed. 

9.2.5  Evaluation  Methods 

Use  the  following  methods  to  evaluate  warning  system  performance. 

9.2.5. 1 Rate  System  Response 

Examine  the  video  and  audio  data  and  rate  each  warning  scenario. 

The  rating  is  a false  positive  if  the  system  responds  with  a warning  while  the  vehicle 
approaches  the  split  while  inside  the  straight  lane  markers. 
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9.2.6  Reporting 

The  evaluator  completes  the  following  tables  from  the  data  in  each  test. 


Name  of  Test 

Curve  Speed  Warning  Test 

Date  of  Test 

Warning  System  ID 

Name  and  Version  Number 

Test  Run  Results 


Factors 

Results 

Condition 

Speed 

mph 

#of 

runs 

Run 

# 

CR 

T/F 

vf 

mph 

Possible  causes  of  false  alarm 

Warm/Dry 

67 

10 

1 

»» 

u 

!» 

2... 

CR  = System  gave  correct  response  (T  if  no  warning)  or  failed  to  give  correct  response 
(F  if  false  alarm). 

Vf  = measured  vehicle  forward  speed 
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Appendix  A Methods  for  Determining  Road  Dimensions  and 
Geometry 

This  section  contains  recommended  methods  to  verify  that  road  segments  satisfy  the 
geometry  specifications  prescribed  for  the  test. 

A.l  Lane  Marker  Dimensions 

Use  a tape  measure  to  verify  lane  marker  dimensions  before  the  test.  Alternatively,  use 
the  IMS  to  measure  lane  marker  dimensions. 

A.2  Lane  Width 

Use  a tape  measure  to  verify  lane  width  before  the  test.  Alternatively,  use  the  IMS  to 
measure  lane  width. 

A.3  Curve  Radius 

Calculate  the  curve  radius  of  the  road  segment  over  the  entire  road  segment  or  over  the 
portion  of  the  road  segmented  or  at  the  point  of  road  departure. 

Calculate  the  curve  radius  from  vehicle  speed  and  a yaw  rate  sensor  (gyro)  using  the 
following  formula  (derived  from  arc  length  formula): 

v 

r - — , 

6' 

Where: 

r = radius  of  curve  (m) 
v = vehicle  forward  speed  (m/s) 

6 ’ = the  yaw  rate  (radians/s). 

If  a GPS  is  used,  calculate  the  curvature  using  a three  points  measured  on  the  curve  using 
the  Independent  Measurement  System  to  visually  measure  road  edge  points. 
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Appendix  B Methods  for  Determining  Lane  Marker 
Characteristics 

This  appendix  provides  standards  for  measuring  and  attaining  color  and  reflectivity 
properties  of  horizontal  pavement  markings  to  ensure  that  lane  markings  provide 
appropriate  conspicuity  for  the  warning  system  sensor.  The  performance  of  pavement 
markings  is  determined  by  the  coefficient  of  retroreflected  luminance,  Rl , the  coefficient 
of  retroreflected  luminance  for  wet  roadways,  RL-wet,  the  coefficient  of  retroreflected 
luminance  under  continuous  wetting,  RL-Rain , or  the  luminance  coefficient  under  diffuse 
illumination,  Qd.  Under  the  same  conditions  of  headlight  illumination  and  driver’s 
viewing,  larger  values  of  these  coefficients  correspond  to  higher  levels  of  visual 
performance  at  corresponding  geometry.  Reapply  the  lane  markings  when  they  measure 
outside  of  the  specified  value. 

The  following  standards  apply  for  lane-marker  reflectance  specifications: 

ASTM  D 6628  -01,  “Standard  Specification  for  Color  of  Pavement  Marking  Materials”, 

ASTM  D 6359  - 99,  “Standard  Specification  for  Minimum  Retroreflectance  of  Newly 
Applied  Pavement  Marking  Using  Portable  Hand-Operated  Instruments”, 

ASTM  E 1710,  “Standard  Test  Method  for  Measurement  of  Retroreflective  Pavement 
Marking  Materials  with  CEN-prescribed  Geometry  Using  a Portable  Retroreflectometer”, 

ASTM  E 21 77,  “Standard  Test  Method  for  Measuring  the  Coefficient  of  Retroreflected 
Luminance  of  Pavement  Markings  in  a Standard  Condition  of  Wetness”, 

ASTM  Designation  E 2176,  “Standard  Test  Method  for  Measuring  the  Coefficient  of 
Retroreflected  Luminance  of  Pavement  Markings  in  a Standard  Condition  of  Continuous 
Wetting”,  and 

European  Standard  EN  1436:1997/prAl:2002  “Road  marking  materials  - Road  Marking 
Performance  for  Road  Users” 

B.l  Terminology 

1 . Luminance  coefficient  under  diffuse  illumination,  Qd  - the  ratio  of  the  luminance,  L, 
of  field  of  the  road  marking  in  the  given  direction  by  the  illuminance  ,E,  on  the  field, 
expressed  in  millicandelas  per  square  meter  per  lux  (mcdnn^lx'1). 

2.  Coefficient  of  retroreflected  luminance,  Rl  - the  ratio  of  the  luminance,  Z,  of  the  field 
of  the  road  marking  in  the  direction  of  observation  by  the  normal  illuminance,  El , 
the  field  perpendicular  to  the  direction  of  the  incident  light,  expressed  in 
millicandelas  per  square  meter  per  lux  (mcd*m'“*lx'  ). 

3.  Checkpoint  area  - the  evaluation  area  within  each  zone  of  measurement. 

4.  Portable  Retroreflectometer  - an  instrument  that  can  be  used  in  the  field  or  laboratory 
for  measuring  the  coefficient  of  reflected  luminance,  Rl  , and  complies  with  Test 
Method  ASTM  E 1710. 

5.  Condition  of  wetness  - the  test  condition  created  by  liberally  wetting  the  road 
marking  and  waiting  a certain  time  period  after  wetting  for  the  water  to  run  off. 
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6.  Coefficient  of  retroreflected  luminance  on  a wet  surface,  R.L-wet  - retroreflectivity  of 
horizontal  pavement  markings  in  a standard  condition  of  wetness.  This  condition 
typically  exists  after  a rainfall  has  ended  and  the  pavement  markings  are  still  wet  or 
as  the  markings  are  still  wet  from  dew  or  humidity. 

7.  Coefficient  of  retroreflected  luminance  on  a continuously  wetted  surface,  Rl-Row  - 
retroreflectivity  of  horizontal  pavement  markings  in  a standard  condition  of  continual 
wetting.  This  condition  typically  exists  during  a rainfall. 

8.  Zone  of  measurement  - the  road  length  to  measure  containing  the  marking  units  that 
appear  similar  in  a visual  inspection. 

B.2  Measurement  Frequency 

The  majority  of  Run-Off-Road  sensor  validation  tests  occur  in  a limited  area  of  a test 
track  or  roadway.  In  such  limited  areas,  the  test  area,  the  zone  of  measurement  and  the 
checkpoint  area  are  equivalent. 

Validate  longer  zones  of  measurement  (>  300  m)  via  two  checkpoint  areas  100  m long 
each  and  separated  by  at  least  100  m for  each  kilometer  of  the  test  zone.  The  initial  and 
final  100  m of  the  test  zone  must  be  included  as  check  point  areas.  Carry  out  the 
necessary  remediation  and  re-verification  procedures  if  a test  zone  fails  to  meet  the 
required  specifications. 

Within  each  checkpoint  area,  measure  reading  sites  approximately  every  5 m. 

Measurements  along  Broken  Longitudinal  Horizontal  Pavement  Markings  shall  be  the 
same  as  for  continuous  markings  except  there  shall  be  two  reading  sites  per  mark  and  no 
readings  between  marks. 

B.3  Measurements 

All  measurements  obtained  in  the  sampling  areas  are  in  the  direction  of  traffic  flow,  i.e., 
in  the  direction  of  the  vehicle  motion  during  the  sensor  test.  Measure  both  single  and 
double  lines  to  determine  compliance  with  prescribed  dimensions  and  the  acceptance 
criteria  applied  individually  to  both. 

The  measurement  geometry  used  to  determine  the  average  coefficient  of  reflective 
luminance  and  the  luminance  coefficient  under  diffuse  illumination  shall  be  88.76°  for 
the  entrance  angle  (Pi),(P2=0°)  and  1.05°  for  the  observation  angle.  The  aperture  angles 
for  both  the  source  (if  appropriate)  and  receiver  shall  not  exceed  0.33°.  These 
measurement  methods  specify  the  use  of  portable  or  mobile  reflectometers  that  can 
measure  pavement  markings  in  accordance  with  ASTM  Test  Method  E 1710  or  CEN 
1436.  The  standard  measurement  condition  (see  Figure  33)  represents  the  angles 
corresponding  to  a distance  of  30  m for  the  driver  of  a passenger  car  with  an  eye  height  of 
1.2  m and  a headlight  height  (if  any)  of  0.65  m above  the  road,  commonly  called  the 
“CEN  30  meter  geometry”. 
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Surface  Normal 


Retroreflective  Road  marking  Stripe 


Figure  33.  CEN  30  Meter  Geometry 


Measurements  shall  be  in  accordance  with  the  retroreflectometer  manufacturer’s 
specification  for  operation  and  calibration.  When  possible  a two-point  calibration  is 
preferred.  The  operator  shall  position  the  retroreflectometer  squarely  on  or  in  front  of  the 
pavement  marking  material  with  the  illumination  in  the  direction  of  the  sensor  test  such 
that  the  illuminated  measurement  area  fits  within  the  width  of  the  road  marking. 

Take  measurements  within  30  days  of  the  Run-Off-Road  sensor  test  when  the  road  is 
clean.  If  using  salt,  dirt,  or  a wetting  agent  to  adjust  the  luminance  of  the  markings,  take 
measurements  within  24  hours  providing  there  is  no  precipitation,  high  winds,  or  road 
cleaning  in  the  test  zone  between  the  luminance  measurements  and  the  Run-Off-Road 
sensor  test. 

B.4  Performance  Requirement 

The  pavement  marking  material  shall  be  retroreflecting  white  or  yellow,  and  shall  be 
readily  visible  as  white  or  yellow  when  viewed  from  the  sensor  position  with  automobile 
headlights  at  night.  Measure  the  pavement  marking  after  removal  of  all  excess  glass 
spheres.  Optionally,  use  light  debris,  such  as  dirt  or  salt,  to  retard  the  marking’s 
reflectivity  to  achieve  a value  within  the  desired  range. 

The  use  of  the  sampling  procedure  and  tables  for  inspection  by  attributes  (see  US  Mil-Std 
105D  “Sampling  Procedure  and  Tables  for  Inspection  by  Attributes”)  forms  the  basis  of 
the  acceptance  criteria.  Measurements  shall  proceed  until  the  decision  point  for  rejection 
is  reached,  in  which  case  the  probability  is  95  % that  more  then  20  % of  the  pavement 
markings  will  fall  below  or  more  then  20  % of  the  pavement  markings  will  fall  above  the 
requirement.  If  less  the  acceptance  number  of  readings  fall  below  or  above  the 
requirements,  the  probability  is  95  % that  less  then  5 % of  the  pavement  marking  will  fall 
below  or  more  then  20  % of  the  pavement  markings  will  fall  above  the  values  specified 
and  the  pavement  marking  is  accepted. 

B.4.1  Dry  Day  Road  Tests 
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The  method  measures  the  wet  retroreflectivity  of  horizontal  pavement  markings. 
Measurements  shall  be  taken  after  a sustained  period  without  precipitation  (rain,  drizzle, 
or  fog)  and  the  roadway  and  markings  appear  dry  to  the  touch,  and  there  is  no  evidence  of 
residual  wetting. 


Measure  the  luminance  coefficient  under  diffuse  illumination,  Qd,  in  accordance  with 
European  Standard  EN1436:  1997  Annex  A.  For  any  set  of  contiguous  20  readings 
(approximately  100  m),  not  more  then  3 readings  may  be  greater  then  130  mcd  nf  /lx, 
and  not  more  then  3 readings  may  be  less  then  100  mcd  m’  /lx  for  the  markings  to  be 
acceptable  for  Run  Off  Road  Sensor  testing. 

B.4.2  Dry  Night  Road  Tests 

The  method  measures  the  wet  retroreflectivity  of  horizontal  pavement  markings. 
Measurements  shall  be  taken  after  a sustained  period  without  precipitation  (rain,  drizzle, 
or  fog)  and  the  roadway  and  markings  appear  dry  to  the  touch,  and  there  is  no  evidence  of 
residual  wetting. 

Measure  the  coefficient  of  retroreflected  luminance  in  accordance  with  ASTM  ASTM  E 
1710.  For  any  set  of  contiguous  20  readings  (approximately  100  m),  not  more  then  3 
readings  may  be  greater  then  130  mcd  m'  /lx,  and  not  more  then  3 readings  may  be  less 
then  1 00  mcd  m~  /lx  for  the  markings  to  be  acceptable  for  Run  Off  Road  Sensor  testing. 

B.4.3  Wet  Road  Tests 

This  method  measures  the  wet  retroreflectivity  of  horizontal  pavement  markings  in  a 
standard  condition  of  wetness.  This  condition  typically  exists  after  a rainfall  has  ended 
and  the  pavement  markings  are  still  wet  or  as  the  markings  are  still  wet  from  dew  or 
humidity.  Measurements  for  tests  involving  wet  roads  are  similar  to  methods  prescribed 
in  ASTM  E 2177.  “Standard  Test  Method  for  Measuring  the  Coefficient  of  Retroreflected 
Luminance  of  Pavement  markings  in  a Standard  Condition  of  Wetness”. 

The  operator  shall  wet  the  target  area  with  a hand  sprayer  or  bucket.  The  target  area 
includes  the  marking  and  the  surrounding  area.  When  using  a hand  sprayer,  wet  the  area 
(the  stripe  and  the  road  surface)  for  30  s.  The  marking  and  road  surface  are  to  be 
completely  flooded.  When  a hand  sprayer  is  not  available,  the  operator  shall  slowly  pour 
approximately  10  liters  of  clean  water  from  a bucket  over  the  measurement  area  (an 
approximately  20  ± 2 in  diameter  circle).  Pour  an  even  crest  of  water  to  temporarily 
flood  the  test  surface  area. 

Measure  the  coefficient  of  retroreflected  luminance  of  the  wetted  marking  45  ± 5 s after 
completion  of  the  wetting.  For  any  set  of  20  readings  (approximately  100  m),  not  more 
then  3 readings  may  be  greater  then  50  mcd  m'  /lx,  and  not  more  then  3 readings  may  be 
less  then  25  mcd  m~  /lx  for  the  markings  to  be  acceptable  for  Run  Off  Road  Sensor 
testing. 

B.4.4  Continuous  Wetting  Tests 

This  method  of  measuring  of  the  wet  retroflective  properties  of  pavement  markings 
utilizes  a method  of  continuously  wetting  the  marking  during  measurement  with  the 
retroreflectometer  and  closely  follows  ASTM  E 2176,  “Standard  Test  Method  for 
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Measuring  the  Coefficient  of  Retroreflected  luminance  of  Pavement  markings  in  a 
Standard  Condition  of  Continuous  Wetting”. 


Steady  State  □ 
Area 


Figure  34  Definition  of  RL-Rain 

The  operator  shall  position  and  adjust  the  water  spray  with  the  nozzle  such  that  it 
provides  an  even  spray  covering  the  whole  area  to  be.  The  spray  shall  cover  an  area  that 
is  25  % greater  then  the  measuring  field  measured  (generally  a 20  ± 2 in  diameter  circle). 
The  variation  between  the  greatest  and  lowest  intensity  within  the  area  shall  not  be 
greater  then  the  ratio  1.7:1.  The  average  spray  shall  be  approximately  20  mm/h  (heavy 
rain). 

The  operator  shall  take  retroreflectometer  readings  every  10  s until  steady  state  occurs 
(commonly  between  30  s and  60  s).  For  any  set  of  20  readings  (approximately  100  m), 
not  more  then  3 readings  may  be  greater  then  50  mcd  m"2/lx,  and  not  more  then  3 
readings  may  be  less  then  25  mcd  m'  /lx  for  the  markings  to  be  acceptable  for  Run  Off 
Road  Sensor  testing. 

B.5  Reporting 

The  test  report  shall  include: 

1 . the  date  and  time, 

2.  the  location  of  the  test  in  the  form  of  roadway  number,  cardinal  points,  linear  distance 
from  the  cardinal  points,  traffic  direction,  and  line  identification, 

3.  the  description  of  the  instrumentation  and  the  test  method  used, 

4.  the  intended  road  condition  (dry,  wet,  wetting), 

5.  the  readings  recorded  in  millicandelas  per  square  meter  per  lux  (mcd*m'“*lx' ). 
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